Azure AI Foundry Deep Dive Series Part 6: Security and Governance Implementation

Security and governance separate successful AI deployments from disasters waiting to happen. Organizations treating these as afterthoughts face data breaches, compliance violations, and eroded trust. But those embedding security from day one build resilient systems that scale confidently. Azure AI Foundry achieved ISO/IEC 42001:2023 certification for AI management systems, validating that Microsoft’s approach to responsible AI meets global standards. This final post in our series covers the comprehensive security architecture, governance frameworks, and compliance patterns that protect AI workloads in production.

The Security Landscape for AI

AI systems face threats traditional applications never encounter. Prompt injection attacks manipulate models into revealing sensitive data or performing unauthorized actions. Model poisoning corrupts training data to introduce backdoors. Data exfiltration through carefully crafted queries extracts information the model shouldn’t share.

These threats require defense in depth. Security operates at multiple layers: network isolation prevents unauthorized access, identity management controls who can use resources, data encryption protects information at rest and in transit, content filtering blocks harmful outputs, and audit logging tracks all activity for forensic analysis.

Azure AI Foundry provides built-in protection against these threats while giving organizations control over security posture. The platform inherits Azure’s zero-trust architecture where no connection is trusted by default. Models run in isolated virtual machines with the same protections that defend against traditional malware.

Network Security Architecture

Network isolation forms the foundation of production AI security. Public internet access to AI resources creates attack surface. Private endpoints eliminate this exposure by keeping traffic within your virtual network.

Private Link Configuration

Configure private endpoints for all Azure AI Foundry resources. This includes the Foundry resource itself, Azure OpenAI endpoints, Azure AI Search, Cosmos DB, and Storage accounts.

Here’s the architecture for a fully private deployment:

flowchart TB
    subgraph Internet
        Users[External Users]
    end
    
    subgraph VNet["Virtual Network (Private)"]
        subgraph FrontendSubnet["Frontend Subnet"]
            AppGW[Application Gateway
WAF Enabled]
            AppService[App Service
VNet Integration]
        end
        
        subgraph AgentSubnet["Agent Subnet"]
            PrivateEP1[Private Endpoint
Foundry Resource]
            PrivateEP2[Private Endpoint
Azure OpenAI]
            PrivateEP3[Private Endpoint
Claude Models]
        end
        
        subgraph DataSubnet["Data Subnet"]
            PrivateEP4[Private Endpoint
Cosmos DB]
            PrivateEP5[Private Endpoint
Storage]
            PrivateEP6[Private Endpoint
AI Search]
        end
        
        subgraph SecuritySubnet["Security Subnet"]
            Firewall[Azure Firewall
Egress Control]
            NSG[Network Security Groups]
        end
    end
    
    subgraph Monitoring["Security Monitoring"]
        Defender[Microsoft Defender
for Cloud]
        Sentinel[Microsoft Sentinel
SIEM]
        Purview[Microsoft Purview
Data Governance]
    end
    
    Users -->|HTTPS| AppGW
    AppGW -->|Inspects Traffic| AppService
    AppService -->|Private Network| PrivateEP1
    PrivateEP1 --> PrivateEP2
    PrivateEP1 --> PrivateEP3
    PrivateEP2 --> PrivateEP4
    PrivateEP2 --> PrivateEP5
    PrivateEP2 --> PrivateEP6
    
    NSG -.->|Controls Traffic| AgentSubnet
    NSG -.->|Controls Traffic| DataSubnet
    
    PrivateEP2 -->|Controlled Egress| Firewall
    
    Defender -.->|Monitors| VNet
    Sentinel -.->|Analyzes| VNet
    Purview -.->|Governs Data| DataSubnet

Implement this configuration with Infrastructure as Code:

resource "azurerm_private_endpoint" "foundry" {
  name                = "foundry-private-endpoint"
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name
  subnet_id           = azurerm_subnet.agent_subnet.id

  private_service_connection {
    name                           = "foundry-connection"
    private_connection_resource_id = azurerm_cognitive_account.foundry.id
    is_manual_connection          = false
    subresource_names             = ["account"]
  }

  private_dns_zone_group {
    name                 = "foundry-dns-zone-group"
    private_dns_zone_ids = [azurerm_private_dns_zone.foundry.id]
  }
}

resource "azurerm_private_dns_zone" "foundry" {
  name                = "privatelink.cognitiveservices.azure.com"
  resource_group_name = azurerm_resource_group.main.name
}

resource "azurerm_private_dns_zone_virtual_network_link" "foundry" {
  name                  = "foundry-dns-link"
  resource_group_name   = azurerm_resource_group.main.name
  private_dns_zone_name = azurerm_private_dns_zone.foundry.name
  virtual_network_id    = azurerm_virtual_network.main.id
}

Network Security Groups

NSGs control traffic flow between subnets. Define restrictive rules that allow only necessary communication:

resource "azurerm_network_security_rule" "agent_to_data" {
  name                        = "AllowAgentToData"
  priority                    = 100
  direction                   = "Outbound"
  access                      = "Allow"
  protocol                    = "Tcp"
  source_port_range          = "*"
  destination_port_range     = "443"
  source_address_prefix      = "10.0.2.0/24"  # Agent subnet
  destination_address_prefix = "10.0.3.0/24"  # Data subnet
  resource_group_name        = azurerm_resource_group.main.name
  network_security_group_name = azurerm_network_security_group.agent.name
}

resource "azurerm_network_security_rule" "deny_internet" {
  name                        = "DenyInternetOutbound"
  priority                    = 4096
  direction                   = "Outbound"
  access                      = "Deny"
  protocol                    = "*"
  source_port_range          = "*"
  destination_port_range     = "*"
  source_address_prefix      = "*"
  destination_address_prefix = "Internet"
  resource_group_name        = azurerm_resource_group.main.name
  network_security_group_name = azurerm_network_security_group.agent.name
}

Managed Network Isolation

Azure AI Foundry’s managed network mode provides hub-scoped network isolation. Once enabled, this setting is irreversible to prevent accidental exposure. All compute resources inherit the network configuration automatically.

New roles like Enterprise Network Connection Approver govern network changes. This ensures that modifications go through proper approval workflows rather than allowing developers to bypass security controls.

Identity and Access Management

Least-privilege access prevents security breaches. Azure RBAC provides granular control over who can perform which operations on AI resources.

Role Assignments

Built-in roles provide appropriate permissions for common scenarios. Azure AI User grants inference access without management permissions. Azure AI Contributor allows model deployment and resource configuration. Cognitive Services OpenAI Contributor enables fine-tuning operations.

Create custom roles when built-in options are too broad:

{
  "Name": "AI Agent Developer",
  "Description": "Can create and manage agents but cannot modify security settings",
  "Actions": [
    "Microsoft.CognitiveServices/accounts/agents/read",
    "Microsoft.CognitiveServices/accounts/agents/write",
    "Microsoft.CognitiveServices/accounts/deployments/read",
    "Microsoft.CognitiveServices/accounts/models/read"
  ],
  "NotActions": [
    "Microsoft.CognitiveServices/accounts/privateEndpointConnections/*",
    "Microsoft.CognitiveServices/accounts/networkAcls/*"
  ],
  "DataActions": [
    "Microsoft.CognitiveServices/accounts/providers/chat/completions/action",
    "Microsoft.CognitiveServices/accounts/providers/embeddings/action"
  ],
  "AssignableScopes": [
    "/subscriptions/{subscription-id}"
  ]
}

Managed Identities

Eliminate credentials from application code using managed identities. System-assigned identities tie directly to resources, automatically cleaning up when resources are deleted. User-assigned identities enable shared identity across multiple resources.

Here’s how applications authenticate using managed identity:

from azure.identity import DefaultAzureCredential
from azure.ai.inference import ChatCompletionsClient

# No credentials in code
credential = DefaultAzureCredential()

endpoint = "https://your-foundry-resource.services.ai.azure.com"
client = ChatCompletionsClient(endpoint=endpoint, credential=credential)

# Client automatically uses managed identity for authentication
response = client.complete(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}]
)

Azure Key Vault Integration

Store secrets, API keys, and certificates in Azure Key Vault. The preview integration allows Foundry projects to pull credentials dynamically without hardcoding them. Applications and agents request sensitive values at runtime, maintaining security while enabling functionality.

import os
from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient

# Connect to Key Vault
credential = DefaultAzureCredential()
vault_url = "https://your-keyvault.vault.azure.net"
client = SecretClient(vault_url=vault_url, credential=credential)

# Retrieve secrets dynamically
api_key = client.get_secret("external-api-key").value
db_password = client.get_secret("database-password").value

# Use secrets without exposing them in code or config
# These never appear in logs or source control

Data Encryption and Protection

Azure AI Foundry encrypts all data by default using Microsoft-managed keys. FIPS 140-2 compliant 256-bit AES encryption protects data at rest. TLS 1.2 or higher encrypts data in transit.

Customer-Managed Keys

Regulatory compliance often requires customer-managed encryption keys (CMK). Store keys in Azure Key Vault and configure Foundry resources to use them:

az cognitiveservices account update \
  --name foundry-resource \
  --resource-group ai-production \
  --encryption KeyVaultProperties='{
    "keyName":"foundry-encryption-key",
    "keyVaultUri":"https://your-keyvault.vault.azure.net",
    "keyVersion":"abc123..."
  }'

CMK enables key rotation, access revocation, and audit logging of all key usage. If keys are deleted or access is revoked, encrypted data becomes inaccessible immediately.

Data Residency Controls

Different deployment types offer different data residency guarantees. Standard deployments process data in the resource’s region, ensuring data never leaves specified geography. DataZone deployments allow processing anywhere within the defined zone (like EU or US) while storing data at rest in the customer-designated geography.

Global deployments optimize for performance by potentially processing requests across regions, though data at rest remains in the designated location. Choose deployment types based on compliance requirements and performance needs.

Content Safety and Responsible AI

Azure AI Content Safety provides real-time screening of prompts and outputs for harmful content. The service detects violence, hate speech, sexual content, and self-harm across text and images.

Content Filtering Configuration

Configure filtering thresholds based on your application’s risk tolerance:

from azure.ai.contentsafety import ContentSafetyClient
from azure.ai.contentsafety.models import AnalyzeTextOptions

client = ContentSafetyClient(endpoint, credential)

# Configure content filters
content_filter_config = {
    "violence": {
        "enabled": True,
        "severity_threshold": "Medium"
    },
    "hate": {
        "enabled": True,
        "severity_threshold": "Low"
    },
    "sexual": {
        "enabled": True,
        "severity_threshold": "Medium"
    },
    "self_harm": {
        "enabled": True,
        "severity_threshold": "Medium"
    }
}

# Analyze content before sending to model
request = AnalyzeTextOptions(text=user_input)
result = client.analyze_text(request)

if result.categoriesAnalysis:
    for category in result.categoriesAnalysis:
        if category.severity >= 4:  # High severity
            # Block request, log incident
            return "Request blocked due to content policy"

Prompt Shields

Prompt shields detect jailbreak attempts and prompt injection attacks. These attacks try to manipulate models into ignoring safety guidelines or revealing sensitive information.

Enable prompt shields on all production deployments. The service analyzes input patterns and blocks suspicious requests before they reach the model.

Protected Material Detection

This feature identifies when model outputs contain copyrighted material. Configure it to detect and prevent reproduction of protected content, reducing legal risk.

Audit Logging and Compliance

Comprehensive logging enables forensic analysis after security incidents and demonstrates compliance with regulatory requirements.

Azure Monitor Integration

Configure diagnostic settings to capture all operations:

az monitor diagnostic-settings create \
  --name foundry-diagnostics \
  --resource /subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.CognitiveServices/accounts/{name} \
  --logs '[
    {
      "category": "Audit",
      "enabled": true
    },
    {
      "category": "RequestResponse",
      "enabled": true
    },
    {
      "category": "Trace",
      "enabled": true
    }
  ]' \
  --metrics '[
    {
      "category": "AllMetrics",
      "enabled": true
    }
  ]' \
  --workspace /subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.OperationalInsights/workspaces/{workspace}

Query logs to investigate security events, track user activity, and generate compliance reports. Log Analytics provides powerful query capabilities for correlation and pattern detection.

Microsoft Sentinel Integration

Microsoft Sentinel provides SIEM capabilities for security monitoring. Connect Azure AI Foundry logs to Sentinel for advanced threat detection:

// Detect unusual API access patterns
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
| where Category == "RequestResponse"
| summarize RequestCount = count() by CallerIpAddress, bin(TimeGenerated, 1h)
| where RequestCount > 1000  // Threshold for unusual activity
| project TimeGenerated, CallerIpAddress, RequestCount
| order by TimeGenerated desc

// Identify failed authentication attempts
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
| where Category == "Audit"
| where ResultSignature == "401"
| summarize FailedAttempts = count() by CallerIpAddress, bin(TimeGenerated, 15m)
| where FailedAttempts > 5
| project TimeGenerated, CallerIpAddress, FailedAttempts

Microsoft Purview Data Governance

Microsoft Purview provides data governance capabilities for AI applications. Enable Purview integration to enforce Data Loss Prevention policies, honor sensitivity labels, and track data lineage.

DLP Policy Enforcement

Prevent AI applications from sharing sensitive information:

from purview_sdk import PurviewClient

purview_client = PurviewClient(endpoint, credential)

# Check if content contains sensitive data before processing
def check_dlp_compliance(content):
    """Verify content doesn't violate DLP policies"""
    result = purview_client.dlp.analyze(content)
    
    if result.has_violations:
        sensitive_types = [v.sensitive_info_type for v in result.violations]
        print(f"DLP violation detected: {sensitive_types}")
        return False
    
    return True

# Use in application
user_prompt = "My SSN is 123-45-6789"
if not check_dlp_compliance(user_prompt):
    return "Request contains sensitive information that cannot be processed"

Sensitivity Labels

Apply sensitivity labels to data used for model training and inference. Purview enforces that applications respect these labels, preventing over-sharing of classified information.

Azure Policy for Governance

Azure Policy enforces organizational standards at scale. Define policies that automatically apply to all AI resources.

Built-In Policies

Apply Microsoft’s recommended policies for AI workloads:

az policy assignment create \
  --name "require-private-endpoints" \
  --display-name "Cognitive Services should use private endpoints" \
  --scope /subscriptions/{subscription-id}/resourceGroups/{resource-group} \
  --policy "/providers/Microsoft.Authorization/policyDefinitions/0fda3595-9f2b-4592-8675-4231d6fa82fe"

az policy assignment create \
  --name "require-cmk" \
  --display-name "Cognitive Services accounts should use customer-managed keys" \
  --scope /subscriptions/{subscription-id}/resourceGroups/{resource-group} \
  --policy "/providers/Microsoft.Authorization/policyDefinitions/67121cc7-ff39-4ab8-b7e3-95b84dab487d"

Custom Policies

Create custom policies for organization-specific requirements:

{
  "mode": "Indexed",
  "policyRule": {
    "if": {
      "allOf": [
        {
          "field": "type",
          "equals": "Microsoft.CognitiveServices/accounts"
        },
        {
          "field": "Microsoft.CognitiveServices/accounts/deployment.model",
          "contains": "preview"
        }
      ]
    },
    "then": {
      "effect": "deny"
    }
  },
  "parameters": {},
  "metadata": {
    "displayName": "Block preview models in production",
    "description": "Prevents deployment of preview models to production resources"
  }
}

Compliance Certifications

Azure AI Foundry maintains extensive compliance certifications. ISO/IEC 42001:2023 certification for AI management systems validates Microsoft’s responsible AI practices. SOC 2 Type II attests to security controls. HIPAA compliance enables healthcare applications. FedRAMP authorization allows government use.

These certifications enable organizations to inherit compliance controls rather than building them from scratch. Map your regulatory requirements to Azure’s certifications and document the shared responsibility model.

Incident Response

Despite preventive controls, security incidents occur. Prepare response procedures before incidents happen.

Define incident severity levels and response times. Critical incidents like data breaches require immediate response within minutes. High-severity incidents like unauthorized access need response within hours. Medium and low-severity incidents can wait for business hours.

Maintain runbooks for common scenarios. Document steps for isolating compromised resources, rotating credentials, analyzing logs, and notifying stakeholders. Practice these procedures through tabletop exercises.

Preserve evidence for forensic analysis. Don’t delete logs or resources until investigation completes. Use Azure Resource locks to prevent accidental deletion during incident response.

Best Practices Summary

Implement private endpoints for all AI resources, eliminating public internet exposure. Use managed identities exclusively, never embedding credentials in code. Enable customer-managed keys for sensitive workloads requiring full control over encryption.

Configure content filtering appropriate to your application’s risk profile. Enable prompt shields on all production deployments. Monitor for unusual access patterns using Sentinel.

Apply Azure Policy to enforce security standards automatically. Tag resources for governance and compliance tracking. Integrate with Purview for data governance.

Document your security architecture and keep it current. Conduct regular security assessments and penetration testing. Update runbooks based on lessons learned from incidents and exercises.

Train development teams on secure AI practices. Security isn’t just infrastructure. It’s also how developers write prompts, handle user input, and process model outputs.

Series Conclusion

This series covered the complete lifecycle of building production AI applications on Azure AI Foundry. We started with architecture fundamentals, progressed through multi-model integration and fine-tuning, optimized costs, and finished with comprehensive security.

Azure AI Foundry provides the building blocks for enterprise AI. Success depends on how you assemble them. The patterns and practices documented here come from hundreds of production deployments across industries. They work.

Start small with pilot projects. Validate architecture decisions before scaling. Measure everything: quality, cost, performance, and security. Iterate based on data rather than assumptions.

The AI landscape evolves rapidly. What’s cutting-edge today becomes baseline tomorrow. Stay current with Azure’s monthly updates. Participate in preview programs for early access to new capabilities. Build systems that adapt as technology advances.

Most importantly, focus on business value. AI is a tool, not a goal. The best technical architecture means nothing if it doesn’t solve real problems. Align AI initiatives with business objectives. Measure success in business terms. Build sustainably for the long term.

References

• Azure security baseline for Azure AI Foundry – Microsoft Learn
• Azure AI Foundry Models achieve ISO/IEC 42001:2023 certification – Azure Blog
• Govern Azure platform services for AI – Cloud Adoption Framework
• Develop secure and compliant AI Apps with Microsoft Purview – Microsoft Learn
• Securing generative AI models with Microsoft Security – Microsoft Security Blog