Log Analytics Workspace Architecture
Building Effective Logging Infrastructure
Introduction
A well-designed Log Analytics workspace is the foundation for effective observability in Azure. It collects, stores, and enables analysis of logs and metrics from across your integration workloads. For Azure Functions, Service Bus, API Management, and other integration components, the workspace architecture determines what you can query, how fast you can investigate, and how much you pay.
This comprehensive guide covers:
- Workspace design — Architecture decisions
- Data sources — What to collect
- Performance — Query optimization
- Cost management — Data retention and pruning
- Security — Access control
Workspace Architecture
Multi-Workspace Design
┌─────────────────────────────────────────────────────────────────────┐
│ LOG ANALYTICS ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ SINGLE WORKSPACE (Simple) │
│ ─────────────────────────── │
│ • One workspace per subscription │
│ • All logs in one place │
│ • Simple, but limited flexibility │
│ • Good for small environments │
│ │
│ MULTI-WORKSPACE (Recommended) │
│ ────────────────────────── │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Production │ │ Staging │ │ Development │ │
│ │ Workspace │ │ Workspace │ │ Workspace │ │
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
│ │ │ │ │
│ └─────────────-──┼──────────────-─┘ │
│ ▼ │
│ ┌─────────────────────┐ │
│ │ Cross-Workspace │ │
│ │ Query/Reporting │ │
│ └─────────────────────┘ │
│ │
│ Benefits: Isolation, cost allocation, per-team access │
│ │
└─────────────────────────────────────────────────────────────────────┘
Workspace Design Decisions
{
"workspaceDesign": {
"defaultWorkspace": {
"name": "la-integration-prod",
"location": "eastus",
"sku": "pergb2018",
"retention": 30,
"dailyCap": 50,
"resourceGroup": "rg-logging"
},
"regionalWorkspaces": [
{
"name": "la-westus2-prod",
"location": "westus2",
"purpose": "West US region data",
"linked": true
}
],
"dataSources": [
"Azure Function Apps",
"Service Bus Namespaces",
"API Management",
"Logic Apps",
"Storage Accounts",
"Key Vault"
]
}
}
Data Collection
Integration Data Sources
# Enable AzureDiagnostics for Service Bus
az monitor diagnostic-settings create \
--name servicebus-diags \
--resource /subscriptions/xxx/resourceGroups/rg-integration/providers/Microsoft.ServiceBus/namespaces/mynamespace \
--workspace /subscriptions/xxx/resourceGroups/rg-logging/providers/Microsoft.OperationalInsights/workspaces/la-integration-prod \
--logs '[
{"category": "Audit", "enabled": true},
{"category": "OperationalLogs", "enabled": true},
{"category": "Metrics", "enabled": true}
]' \
--metrics '[
{"category": "AllMetrics", "enabled": true}
]'
# Enable for Azure Functions
az monitor diagnostic-settings create \
--name functions-diags \
--resource /subscriptions/xxx/resourceGroups/rg-integration/providers/Microsoft.Web/sites/myfunctionapp \
--workspace /subscriptions/xxx/resourceGroups/rg-logging/providers/Microsoft.OperationalInsights/workspaces/la-integration-prod \
--logs '[
{"category": "FunctionAppLogs", "enabled": true},
{"category": "AppServiceHTTLogs", "enabled": true}
]'
Log Types
{
"logTypes": {
"applicationLogs": {
"source": "Azure Functions, Logic Apps",
"tables": "AppEvents, AppTraces",
"retention": 30,
"cost": "Standard"
},
"platformLogs": {
"source": "Azure Resource Manager",
"tables": "AzureDiagnostics, Perf",
"retention": 90,
"cost": "Standard"
},
"auditLogs": {
"source": "Azure AD, Resource changes",
"tables": "AuditLogs, AzureActivity",
"retention": 365,
"cost": "Standard"
},
"customLogs": {
"source": "Application code",
"tables": "CustomLogs, CustomEvents",
"retention": 30,
"cost": "Standard"
}
}
}
Query Optimization
Performance Best Practices
┌─────────────────────────────────────────────────────────────────────┐
│ QUERY OPTIMIZATION │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ DO: │
│ ✓ Use specific table names │
│ ✓ Filter early (where clause first) │
│ ✓ Use datetime filters │
│ ✓ Select only needed columns │
│ ✓ Use parse instead of regex when possible │
│ ✓ Create saved functions for common queries │
│ │
│ DON'T: │
│ ✗ Search on large string fields without filters │
│ ✗ Use wildcards at start of search │
│ ✗ Apply functions to entire table before filtering │
│ ✗ Query without time filter │
│ ✗ Use regex when simpler parsing works │
│ │
└─────────────────────────────────────────────────────────────────────┘
Common Queries
// Service Bus message processing errors
ServiceBusDiagnostics
| where TimeGenerated > ago(1h)
| where OperationName == "SDK.Receive"
| where SeverityLevel == "Error"
| project TimeGenerated, QueueName, ErrorMessage, MessageId
// Function execution failures
AppTraces
| where TimeGenerated > ago(1h)
| where SeverityLevel == "Error"
| where Message contains "Exception"
| extend FunctionName = tostring(parse_json(RowId)["function"])
| summarize Errors = count() by FunctionName
// API Management request errors
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.APIM"
| where OperationName == "ClientAuthentication"
| summarize Errors = count() by ApiName, UserAgent
// Integration flow latency
Perf
| where TimeGenerated > ago(1h)
| where ObjectName == "Function App"
| where CounterName == "Average Response Time"
| summarize AvgLatency = avg(CounterValue) by InstanceName
Cost Management
Data Retention and Pruning
{
"costManagement": {
"workspace": {
"retention": 30,
"archiveRetention": 1,
"dailyCap": {
"enabled": true,
"capGB": 50,
"alertWhenApproaching": 40
}
},
"tables": {
"AppTraces": {
"retention": 7,
"archive": true
},
"Perf": {
"retention": 14,
"archive": false
},
"AuditLogs": {
"retention": 365,
"archive": true
}
},
"optimization": {
"basicLogs": {
"enabled": "Dev/Test environments",
"retention": 7
},
"dataExport": {
"coldStorage": "Long-term retention, compliance"
}
}
}
}
Data Collection Limits
# Set daily cap
az monitor log-analytics workspace update \
--resource-group rg-logging \
--workspace-name la-integration-prod \
--daily-cap 50
# Check usage
az monitor log-analytics workspace show \
--resource-group rg-logging \
--workspace-name la-integration-prod \
--query "dataRetention, dailyQuotaGb, dailyRate"
Access Control
RBAC for Workspaces
# Grant reader access to team
az role assignment create \
--assignee team-group \
--role "Log Analytics Reader" \
--scope /subscriptions/xxx/resourceGroups/rg-logging/providers/Microsoft.OperationalInsights/workspaces/la-integration-prod
# Grant contributor to platform team
az role assignment create \
--assignee platform-team \
--role "Log Analytics Contributor" \
--scope /subscriptions/xxx/resourceGroups/rg-logging/providers/Microsoft.OperationalInsights/workspaces/la-integration-prod
Best Practices
Implementation Checklist
| Practice | Description |
|---|---|
| Use multiple workspaces | Isolate by environment/team |
| Enable daily cap | Prevent runaway costs |
| Configure retention | Match business requirements |
| Optimize queries | Follow performance guidelines |
| Use basic logs | For dev/test environments |
| Monitor usage | Track data volume trends |
Query Performance Tips
┌─────────────────────────────────────────────────────────────────────┐
│ QUERY PERFORMANCE TIPS │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ BEFORE FILTERING: 10M+ rows │
│ ├── Use time filter: | where TimeGenerated > ago(1h) │
│ ├── Use resource filter: | where _ResourceId contains "func" │
│ └── Use specific table: AppTraces │
│ │
│ AFTER FILTERING: 100-1000 rows │
│ ├── Aggregate: | summarize count() by ... │
│ ├── Parse: | parse where ... │
│ └── Project: | project TimeGenerated, Message │
│ │
└─────────────────────────────────────────────────────────────────────┘
Related Topics
- Distributed Tracing — Observability
- Azure Monitor Alerting — Alert configuration
- SLI/SLO/SLA — Service levels
Azure Integration Hub - Architect Level Observability & Operations at Scale