AWS Organizations Migration: Preserving RAM Shares & Lake Formation

Introduction to AWS Organizations Migrations

Restructuring account setups in the cloud is something businesses often face. Mergers, divestitures, or strategic reorganizations can drive the need to move AWS accounts between organizations. But it's not always straightforward, especially when AWS Resource Access Manager (AWS RAM) resource shares and AWS Lake Formation permissions are in play. One big issue is that organization-bound shares trust an account through its organization membership. If an account leaves, AWS RAM kicks out that association. Production workloads that depend on shared resources can get interrupted, so having a continuity plan is pretty important before any account transitions.

A global payment processor managed a migration of 382 AWS accounts, preserving crucial permissions and shares. They used a strategic method in partnership with AWS, and I'll dive into how they pulled it off.

Understanding the Challenge

Moving AWS accounts across organizations makes organization-bound resource shares lose their grip. For production environments, keeping things running smoothly is crucial. The payment processor had to maintain access to shared resources since their operations connected merchants and financial institutions worldwide. While most workloads kept going when original shares dropped, the control plane access was lost. Engineers couldn't tweak shared resources, which could mess with service management and configuration updates.

They had a ticking clock too, with a Transitional Service Agreement (TSA) deadline in April 2026. It got even trickier with their embedded payments platform, a key part of their SaaS offerings, which needed shared databases and tables across 10 accounts. The migration had to keep these dependencies intact without a hiccup.

The Bridge-Share Pattern

To tackle the migration issues, the company used a bridge-share pattern. Before shifting an account, they built a temporary external association, or "bridge," to keep access alive. This involved crafting a bridge share as an external association pre-move, keeping access through the organizational shift. After the move, they reverted the account to its original share for stable permission, avoiding leftover temporary permissions. This method prevents duplicate permission states and keeps the original share as the go-to reference.

AWS gave this pattern a thumbs up, and the company developed automation to ease the process. There are five key steps involved: inventory, create and accept bridges, migrate, restore originals, and validate and remove bridges. Each step plays a role in ensuring permission continuity and integrity during the migration.

Implementation Details

In the implementation phase, the cloud engineering team executed multiple rounds, including 14 non-production waves, a production pilot, 12 weekly production waves, and three contingency waves. Here's a peek into the process:

• Inventory: This involves mapping out original shares, resources, principals, permissions, and regions. It's essential since AWS RAM associations are regional, and a detailed mapping catches all dependencies.
• Create and Accept Bridges: Establish a retained share and accept invitations before migration. This ensures an external association survives the move.
• Migrate: Move the accounts while keeping access through the bridge. There's no need for a standalone period, simplifying things.
• Restore Originals: Get durable shares back with fresh grants during migration. This way, any changes in the window apply to the original share.
• Validate and Remove Bridges: Check active associations before deleting bridges. Automation only drops a bridge after confirming an original share with full coverage.

Thanks to this method, there were no data plane hiccups. Services like AWS Transit Gateway and Amazon Route 53 Resolver chugged along without a hitch.

Why the Bridge Works

This bridge-share pattern takes advantage of AWS RAM's capability to hold onto external associations that outlive organizational changes. By setting up a temporary bridge, the company ensured control-plane access stayed solid throughout the migration. It's particularly crucial for resources like AWS Transit Gateway and AWS Glue Data Catalog that lose control-plane access when shares break. Bridge-share validation is a must before shifting them.

My Engineering Perspective

From my standpoint, having a well-structured approach to account migrations is vital, especially for big global players. Planning and automation can't be overstated. The bridge-share pattern, combined with thorough validation and inventory checks, offers a solid framework for service continuity during migrations. Testing environments should mirror production boundaries to catch potential failures early.

Practical Steps for Implementation

Planning a similar migration? Here’s what I'd recommend:

• Start with a detailed inventory of all shared resources and permissions. AWS RAM associations are regional, so missing one can cause unexpected access issues.
• Ensure your test environments match production boundaries to spot failures early. Those 14 clean waves were great for validation but won't catch production-only issues, so accurate testing is a must.
• Use AWS EventBridge and CloudTrail to oversee resource share changes, which helps in spotting potential issues beforehand.
• Engage with AWS support and utilize documentation like "Transfer AWS accounts between AWS Organizations while preserving AWS Lake Formation permissions." These resources can offer valuable insights.
• Implement automation to handle the creation, validation, and elimination of bridge shares, reducing human error for a consistent process.

These steps, along with a comprehensive understanding of your organization's unique needs, will aid in executing a smooth migration. Keep an eye on dependencies and assess how the migration affects them to ensure service continuity.

Conclusion and Lessons Learned

The payment processor's migration shows how the bridge-share pattern can preserve AWS RAM shares and AWS Lake Formation permissions. Matching validation boundaries with production, monitoring control plane changes, and assessing dependencies early are key takeaways. Experiencing failed terraform apply attempts against a shared AWS Transit Gateway highlights the need for thorough testing and validation.

Success in these migrations rests on detailed planning, automation, and collaborating with AWS. Organizations can take cues from this case study to lessen disruptions and maintain service integrity in AWS Organizations migrations.

For more insights on cloud migration and management, check out our related articles on AWS Glue 6.0 and AWS-Bench.