Case Study

QNET Cloud Migration Research performed by ISFCS

Advising Large Enterprise Cloud Migrations

The client

Center for Medicare and Medicaid Information Systems Group

QNET AWS Cloud-to-Cloud Migration

Challenge

Since 2017, the Center for Clinical Standards and Quality (CCSQ) at the Centers for Medicare and Medicaid Services(CMS) had been operating in two separate Amazon Web Services (AWS) Cloud environments since their initial migration to the AWS Cloud. The management and infrastructure cost of supporting two separate AWS ecosystems (~100 different AWS accounts in one and ~500 different AWS accounts in the other) had proven to be too expensive to support from an infrastructure management and cost perspective.

In the Summer of 2022, CCSQ sought Flexion’s insight because of our technical experts on the proven Information Systems Foundational Components Support team which Flexion has supported since its inception. CCSQ needed an assessment of the risks and opportunities associated with a large-scale enterprise migration from one cloud environment to another, a plan to perform this migration, and any other appropriate recommendations for consideration.

Our approach

Flexion was assigned a high-priority, confidential task to research and plan a cloud-to-cloud migration within CMS. The team analyzed risks and opportunities in migrating a large AWS environment to a new setup. With these insights, they crafted a high-level migration strategy tailored for senior CMS stakeholders. This work was restricted to select CMS personnel, leaving others on a need-to-know basis.

The project presented significant technical and social challenges, due to previous migration issues, requiring detailed research on high-level concepts and careful relationship-building—all while working under strict communication constraints. High-ranking CMS stakeholders required frequent updates on our progress, with tight deadlines, compliance and security concerns, and a need to balance emotional sensitivities—all to ensure a quality-driven, successful cloud-to-cloud migration plan.

From Chaos to Complexity: Separating the What from the How
To tackle the complex cloud-to-cloud migration within stakeholder constraints, we wrangled chaos together to create a structure with a flexible timeline, regular checkpoints, and collaborative research, resulting in a highly praised risks and opportunities analysis and a migration plan that exceeded expectations.

We structured the migration project into two phases: a risk and opportunities analysis followed by a high-level migration strategy. With a collaborative, cross-functional team and ongoing feedback from stakeholders, we worked without defined roles, ensuring thorough research and planning aligned with leadership’s expectations.

Being skeptical and curious to create Risks and Opportunities
The team’s objective was to identify risks in the cloud-to-cloud migration and pinpoint key differences between the environments to streamline the new setup. We conducted rigorous analysis on topics such as IAM administration, data migration, CI/CD, and operational protocols, leveraging CMS’s risk-scoring methodology to assess business impact and ensuring everything was in common language. Mitigation strategies were developed for each risk, and findings were documented in Confluence for continuous feedback from stakeholders. A high-level executive summary with visuals was presented to leadership.

We used the following image to effectively communicate risks and opportunities and received positive feedback to advance the project We utilized expertly designed tables, charts, and diagrams to better showcase the information in a user-friendly format.

The items with risk and without risk were identified for CMS consumption. The items with risk are identified in a table with a header of Category, Technologies, and Aggregated scores based off of CMS Risk Scoring for unmitigated and mitigated. There are 4 rows in the table. Row 1 has Category "Building Software" with technologies "AMI, CI/CD, IAC Maturity, and IAM Management". The unmitigated score is Medium and the mitigated score is Low.Row 2 has Category "Cloud Management" with technologies "Operation, Support Level, and Networking". The unmitigated score is Medium and the mitigated score is Low. Row 3 has Category "Data" with technologies "Data Migration". The unmitigated score is High and the mitigated score is Medium. Row 4 has Category "Technical Cost" with technologies "AWS RI/SP". The unmitigated score is Medium and the mitigated score is Low. The items without risk are identified in a bulleted list with entries "Engineer and Account Onboarding, Available AWS Services, Identity Provider

Image 1: Risk Outline

Our team tackled the complex cloud-to-cloud migration project by systematically analyzing risks, collaborating across teams, and keeping stakeholders informed at every stage.

  • We conducted consistent risk analysis, scoring, and mitigation to group technology categories, offering leadership insights needed for an informed migration strategy.
  • Collaborating with various teams and vendors, we researched each technology area in-depth, scoring findings based on CMS Risk Assessment guidelines to communicate the impact.
  • We shared regular progress updates and findings with CMS leadership and front-office members, ensuring transparency and alignment on project impact and outcomes.

After a thorough risk and opportunity analysis, we developed a high-level migration strategy to guide the transition between cloud environments while addressing all identified risks. Using a collaborative approach, we structured the plan in a shared document with sections on migration strategies, key considerations, and recommendations, regularly reviewing each part with team members and stakeholders. Weekly meetings with AWS allowed us to gather SME support and refine technical specifics essential for the migration, such as Reserved Instances and Savings Plans.

1. A blue-green strategy with traffic routing would have two up-and-running environments. Traffic from the “old” account would be slowly moved to the “new” account to keep production systems running. See “Image 2”.

The design for a Blue Green migration with traffic routing is as follows. Starting from the beginning, the user and external system traffic is routed to the QNET Cloud's Applications and Databases while nothing is sent to the CMS Cloud Applications and Databases while the CMS Cloud is being developed. When the applications in CMS are ready, the traffic will be split from the QNET Cloud environment and the CMS Cloud environment while data is synchronized between them. The majority of the traffic will be sent to the QNET Cloud at first to test out the functionality in CMS Cloud. As confidence is gained in the new environment, then the traffic will be primarily sent to CMS Cloud while only some will be sent to QNET cloud during the decommissioning of the QNET environment. When the QNET environment is fully decommissioned, all traffic will be sent to CMS Cloud.

Image 2: Blue-green migration with traffic routing

2. A second approach would be a blue-green strategy without the traffic routing. It would be a full migration by turning off the “old” environment in favor of the “new” one. See “Image 3”.

The design for a Blue Green migration with traffic routing is as follows. Starting from the beginning, the user and external system traffic is routed to the QNET Cloud's Applications and Databases while nothing is sent to the CMS Cloud Applications and Databases while the CMS Cloud is being developed. When the applications in CMS are ready, the traffic will be split from the QNET Cloud environment and the CMS Cloud environment while data is synchronized between them. The majority of the traffic will be sent to the QNET Cloud at first to test out the functionality in CMS Cloud. As confidence is gained in the new environment, then the traffic will be primarily sent to CMS Cloud while only some will be sent to QNET cloud during the decommissioning of the QNET environment. When the QNET environment is fully decommissioned, all traffic will be sent to CMS Cloud.

Image 3: Blue-green Migration without traffic routing

The plan outlined the migration order, timeline estimates, and considerations across business, security, and technology, highlighting key risks and recommended mitigation strategies. Presented to government leadership, our final report received positive feedback, showcasing our comprehensive research, migration strategy, and expert recommendations, culminating in a successful customer experience.

Technology stack

Enterprise System/ Appliances

Splunk, Jenkins, Ansible Tower
Packer, OKTA, Saviynt
SAS, Active Directory, Github
Nexus, Nessus, F5
Barracuda, Red Hat

Cloud Services

Redshift, RDS, Dynamo
VPC, EC2, ECS
Lambda, EBS, S3
EMR, IAM, Route53
Glue, EKS

Migration Services

DMS
AMS

Outcomes

Our assessment and recommendations received extremely positive reviews from our customers describing it as “very thorough”, “detailed”, and “thoughtful documents”, and that we “exceeded expectations”. We were able to provide the necessary documentation for all the risks and opportunities for the migration as well as an organizational plan on how to proceed. This documentation helped CMS gain confidence and a full understanding of all the pieces necessary to migrate in order to reduce cost, consolidate environments, reduce management overhead, and combine different development environments into one to create a larger engineering community.

Ready to change the way you’re doing business?

Contact us to talk about how Flexion can help your organization drive efficiency, optimize costs, and achieve your technology goals!

A proud AWS partner.

AWS Select Tier Services Partner
Privacy Preferences

When you visit our website, we store information through your browser from specific services, usually in the form of cookies. Feel free to change your Privacy preferences now:

Click to enable/disable Google Analytics tracking code.
Click to enable/disable Google Fonts.

You can also adjust your privacy preferences at any time by visiting the Privacy Policy. Blocking some types of cookies may impact your experience on our website.

Google Analytics tracking is disabled by default, but you can help us understand and improve your experience by enabling it.