Project Summary

Modernized an inherited Azure data platform spanning two actively used subscriptions, replacing fragmented networking, broad network exposure, inconsistent governance, and public service paths with a standardized hub-spoke architecture built around Private Endpoints, centralized DNS, network segmentation, Azure Policy, centralized logging, and tested infrastructure recovery.

The standardization effort raised Microsoft Defender for Cloud’s Secure Score from the mid-20% range to the mid-90% range, reduced permitted inbound network scope by roughly 97%, and established repeatable standards for networking, governance, cost attribution, monitoring, and disaster recovery. Secure Score fluctuates modestly following routine weekend patching cycles.

Problem

Two Azure subscriptions supporting an enterprise data management environment had originally been deployed by an external implementation partner. On paper it looked straightforward: projects were separated into resource groups and provisioned through Terraform. In practice, each project had its own independently provisioned virtual network, with little common architecture between them. Network Security Groups were largely absent, and tagging consisted mostly of Terraform-generated metadata that meant little outside the state file. Ownership, purpose, dependency relationships, and data flows were difficult to determine just by looking at the environment.

Network address space had been allocated defensively, far beyond actual workload requirements. Inbound access followed the same pattern, permitting traffic from large portions of the parent organization’s network rather than the relatively small number of trusted sources that actually needed it.

The more significant issue was service-to-service communication. Resources that should have talked to each other privately were frequently reaching one another over public Azure service endpoints instead. TLS protected that traffic in transit, but encryption alone doesn’t provide the same isolation as keeping communication entirely within private Azure networking. The architecture leaned on public endpoints, IP allowlists, and perimeter controls in places where private connectivity could have substantially reduced the exposed surface. The environment worked, but it didn’t have a coherent platform architecture behind it.

Constraints

• The subscriptions supported active production and development workloads, so modernization couldn’t disrupt existing data pipelines or application dependencies.
• There was no authoritative dependency map describing how resources, pipelines, networks, storage, and databases interacted.
• Network changes required coordination with enterprise networking teams, since inbound connectivity originated outside the Azure subscriptions themselves.
• Terraform was already part of the deployment process and needed to remain compatible with the resulting standards rather than being replaced outright.
• SQL workloads handled regulated institutional data, so relevant configurations and Azure Policy findings had to be reviewed against FERPA-related requirements, not just generic cloud-security benchmarks.
• Existing applications couldn’t be redesigned around the new network. The platform had to be modernized around workloads already in production.

Discovery and Assessment

The first phase was discovery. Several months went into tracing Azure Data Factory pipelines, network relationships, DNS behavior, storage dependencies, database connections, and application traffic to reconstruct the dependency map that should have existed as platform documentation from the start. That process made it clear that manual investigation wouldn’t scale, so purpose-built PowerShell and Python tooling was developed to inventory network topology, resource relationships, Private DNS zones, and configuration state directly from Azure instead.

The original formal finding behind the project was comparatively simple: the allocated address space was too large. Taken literally, that could have been resolved by shrinking a few subnets. But the oversized network turned out to be a symptom of a broader problem: disconnected project networks, loosely defined trust boundaries, excessive address allocation, public service paths, and very little centralized governance. Recognizing that distinction is what turned a subnet-sizing exercise into a full platform redesign.

Architecture

Hub-Spoke Networking

The individual project-network model was replaced with a centralized hub-spoke architecture. The hub became the common control point for connectivity and Private DNS, while project workloads stayed logically separated within spoke networks. Peering relationships became deliberate and centrally understandable instead of emerging independently from each project. That turned the network into something that could be operated as a platform rather than a collection of unrelated deployments.

Private Service Connectivity

Storage accounts and Azure SQL services were first restricted to known network scopes, then migrated to Azure Private Endpoints where cost was feasible. The goal went beyond a narrower public allowlist: removing routine service traffic from public endpoints entirely wherever private Azure connectivity was available, reducing dependence on perimeter filtering and keeping internal service-to-service traffic on private network paths.

Centralized Private DNS

Independent Private DNS configurations were consolidated into hub-managed zones linked to the appropriate spoke networks. This mattered beyond tidiness: Private Endpoints depend on correct private name resolution, and centralizing the zones gave the environment one authoritative location for those records instead of several independent zones each prone to duplication, conflicting records, and drift.

Network Segmentation

Network Security Groups were introduced to establish real segmentation and explicitly control traffic between network boundaries, replacing a model that had relied primarily on broad upstream network access. Connectivity could now be evaluated by source, destination, workload, and actual application requirement.

Governance and Azure Policy

Azure Policy was introduced to move configuration assessment from periodic manual review to continuous governance. Findings were evaluated individually rather than remediated indiscriminately at scale, and relevant findings and SQL configurations were reviewed against FERPA-related requirements. Remediation reflected the actual data and regulatory context of the environment instead of treating every recommendation as an abstract benchmark.

Logging and Operational Visibility

Logging had previously been inconsistent across resources, and in some cases absent entirely. Monitoring and diagnostic information was centralized across both subscriptions, giving the environment a common operational view for troubleshooting, security analysis, and platform support.

Tagging and Cost Governance

Tagging was redesigned so Azure resources could be understood without needing Terraform state as a decoder. Standards were established for identifying ownership, project, environment, and purpose, which also became the basis for attributing Azure spend by project and team, backed by automation to keep the standard enforced rather than relying on administrators remembering to apply it manually.

Disaster Recovery

The environment previously had no meaningful infrastructure recovery capability. Resource configurations were captured using Bicep and JSON-based infrastructure definitions, but those exports weren’t treated as backups just because files existed. Restoration was actually tested: resources were rebuilt from the captured definitions to surface dependency issues, deployment ordering problems, configuration that didn’t round-trip cleanly, and resource types that needed special handling. That testing is what turned configuration exports into a recovery process worth trusting.

Engineering Challenges

Reducing Network Scope Without Breaking Production

The existing network model permitted inbound connectivity from an address range large enough to represent much of the parent enterprise network. Determining what could safely be removed meant identifying the sources genuinely in use, not assuming everything the existing configuration permitted was actually necessary. Working with the enterprise network team, inbound access was narrowed to trusted VPN and designated operational network ranges. That cut the permitted source address space by roughly 97%, down to about eight discrete /24 ranges from the original, much broader allocation.

Reconstructing an Undocumented Environment

There was no authoritative map showing which networks, pipelines, databases, and storage services were actually load-bearing, so a question as simple as “is this network range still required?” couldn’t be answered from documentation. It could only be answered by tracing real workloads, configurations, and traffic. The inventory and automation built to answer that question ended up outlasting the immediate redesign, since the environment could subsequently be assessed programmatically instead of reverse-engineered from scratch every time.

Migrating Live Workloads to Private Endpoints

Moving a resource from public to private connectivity means more than an endpoint configuration change: DNS, network routing, application dependencies, and access controls all have to change in the right sequence, and a Private Endpoint with an incorrect DNS record can make a previously reachable production resource appear to disappear entirely. That’s why DNS consolidation happened as part of the Private Endpoint migration rather than as cleanup afterward: centralizing private resolution reduced the number of moving pieces during migration and gave future deployments a consistent model to follow.

Introducing Policy Into an Existing Environment

Applying Azure Policy to an environment that had never been continuously assessed surfaced a significant backlog of recommendations and configuration gaps at once. Enforcing all of it automatically would have created unacceptable operational risk, so findings were triaged and remediated individually, weighing security improvement against actual workload dependencies. That mattered even more for the data platforms specifically: a configuration that passed a generic cloud benchmark still had to be checked against the regulatory requirements tied to the data it actually held.

Proving Infrastructure Recovery

Exporting infrastructure configuration was the easy part; proving it could actually rebuild the environment was not. Not every Azure resource exports and redeploys cleanly, resources carry dependencies and deployment-sequencing requirements, and some configuration simply doesn’t round-trip without manual adjustment. Repeated restoration testing is what surfaced those gaps and established exactly what needed to be captured, modified, or documented for the environment to be genuinely recoverable.

Results

The redesign produced measurable improvements across security, networking, governance, operations, and recoverability:

• Improved Microsoft Defender for Cloud security posture score from the mid-20% range to approximately 95% in production and 94% in development.
• Reached approximately 96% operational-excellence and 99% performance posture scores, with no active attack-path findings identified.
• Reduced permitted inbound network address space by roughly 97%, replacing organization-scale network access with about eight targeted /24 source ranges tied to actual operational requirements.
• Migrated Azure Storage and SQL workloads from public service endpoints to Private Endpoints where feasible, reducing exposure and dependence on public endpoint allowlists.
• Established hub-spoke networking and NSG-based segmentation where project networks had previously operated largely independently.
• Consolidated project-specific Private DNS configurations into centralized, hub-managed zones, cutting duplication, drift, and troubleshooting complexity.
• Introduced continuous governance through Azure Policy, with relevant findings and SQL configurations reviewed against FERPA-related requirements rather than generic recommendations alone.
• Centralized monitoring and diagnostic logging across both subscriptions for consistent operational and security visibility.
• Standardized resource tagging for team and project cost attribution, supported by automation rather than convention alone.
• Established a disaster-recovery capability using Bicep and JSON configuration capture with tested restoration, replacing an environment where infrastructure recovery had never been validated.
• Replaced an environment that required extensive manual investigation to understand with one where ownership, purpose, network relationships, and operational context are directly visible and repeatable.

The most important outcome was that the environment moved from a collection of independently deployed project resources to an operable platform: privately connected, segmented, governed, measurable, recoverable, and understandable by teams beyond the people who originally built it.

Technologies

Azure Architecture & Networking

Azure Virtual Network, Hub-Spoke Networking, VNet Peering, Network Security Groups, Azure Private Endpoints, Azure Private DNS

Security & Governance

Microsoft Defender for Cloud, Azure Policy, RBAC, network segmentation, private connectivity, FERPA-related configuration review

Monitoring & Operations

Azure Monitor, Log Analytics, Azure Resource Manager, resource tagging, cost attribution

Data Platform

Azure Storage, Azure SQL, Azure Data Factory

Infrastructure & Automation

Terraform, Bicep, ARM/JSON templates, PowerShell, Python