Approach for Migrating from an Existing Middleware Platform to AtomSphere

Document created by Adam Arrowsmith Employee on Dec 16, 2015Last modified by Adam Arrowsmith Employee on Apr 19, 2016
Version 4Show Document
  • View in full screen mode

 

 

Overview

This document provides a high-level approach for migrating interfaces from an existing middleware platform to Dell Boomi AtomSphere. It discusses the unique opportunities and challenges when migrating platforms and offers guidance for dividing the effort into various project phases. However this document is not intended to serve as a detailed project plan.

 

Migrating from Another Middleware Platform

In many ways a migration project from an existing integration platform is similar to a net-new integration project, but there are differences. It is important to be aware of both the advantages and challenges of a migration project to seize improvement opportunities and avoid operational disruption.

 

Advantages

  • Familiarity with use cases - The team has familiarity with integration use cases, end applications, and known issues. Integration management procedures are typically in place and (hopefully!) some interface documentation exists.
  • Known scope - Scope is generally well-defined, although there could be a large number of interfaces to move.
  • Historical data exists - Historical information exists for sample data, volumes and performance as a basis for comparison. However be aware exact metrics may differ in comparison to AtomSphere depending on design.
  • Management tooling - Supporting management tooling likely exists such as reporting dashboards, alerting channels, system monitoring hooks, documentation, change control procedures.
  • Improvement opportunity - The migration project presents an opportunity to optimize integration designs such as streamlining “bolt-on” changes over the years, consolidating and parameterizing similar functionality, and deprecating outdated logic and interfaces.

 

Challenges

  • Learning curve - AtomSphere greatly reduces the learning curve and complexity of middleware platforms and integration development, but it is still a new tool for the team to adopt.
  • New integration approaches - All middleware platforms aim to solve the same general integration problems, but each tool will approach the problem different--AtomSphere is no exception. Interface design will typically need to evolve from the legacy system to take advantage of the architecture and capabilities of AtomSphere. Because of fundamental differences in approach, rebuilding is generally required and usually more efficient vs. migrating legacy assets. The Dell Boomi professional services team can assist in helping translate legacy implementations into AtomSphere designs.
  • Maintain production operations - The cutover must be carefully coordinated with existed interfaces to minimize disruption to production operations.
  • Avoid impact to clients - Impact to existing clients should be avoided/minimized. Ideally the migration would be transparent to client applications but usually that is not entirely possible. Assess impact to client applications (especially those outside the organization) as early as possible. Common changes can include URL endpoints, authentication schemes and credentials, certificates, and whitelisted IPs/hosts. Data formats should be preserved to avoid more costly parsing logic changes for clients.

 

Grouping Interfaces

There are several common ways to organize the interfaces into logical groups for migration. The best grouping is the that makes the most sense for your project and should take into account the number of interfaces and endpoints, commonality across interfaces, project timelines, and external dependencies.

  • Endpoint - Interfaces connecting to/from a given endpoint or set of endpoint applications.
  • Business process - Interfaces enabling a given business process such as order-to-cash or customer-sync.
  • Technical pattern/style - Interfaces using a given integration style such as batch vs. real time, or involve a specific processing requirements. See Technical Patterns below.
  • Criticality - Interfaces supporting critical, frequently used, automated processes vs. less critical processes.
  • Individual interface - Grouping by individual interfaces with no other discernible commonality.

 

Technical Patterns

Thinking in terms of technical patterns can be a great way to better understand the nature of the interfaces and break them down into individual solvable pieces.

 

A pattern-based approach to migrating interfaces to AtomSphere can help organize the work effort, ease the transition, and minimize risk. The big idea is to:

  1. Survey existing interfaces to identify common patterns and/or endpoints
  2. Group interfaces by pattern
  3. Create templates for each pattern
  4. Implement and cutover interfaces by pattern

 

Patterns can be generic or specific, simple or complex, functional or nonfunctional, but typically not end-to-end workflows. Some examples:

  • Endpoint connectivity application/protocol (e.g. SAP, MQ, database, REST) and API interaction (e.g. specific queries/joins, upserts, etc.)
  • Interoperability with existing integration infrastructure components such as ESB, monitoring channels, job tracking dashboards
  • Authentication scheme (especially when publishing web services)
  • Notable transformation or validation requirements (e.g. validate against custom database of allowable values)
  • Integration “style” (scheduled vs. event-based, web services vs. messaging, synchronous, vs. asynchronous.)

 

The goals for cataloging and testing technical patterns are:

  • Identifying external changes required (most notably for real time interfaces such as web services)
  • Assessing level of effort and clarity of requirements for prioritization
  • Discovering opportunities for reuse
  • Gaining experience with the AtomSphere platform

 

Detailed Work Breakdown

 

Migration Phases Overview

The migration effort can be organized into several phases, each with a handful of parallel tracks. When migrating a large number of interfaces, it can make sense to divide the interface-related work (such as requirements gathering, development, and testing) into multiple waves. These waves can be tackled in parallel or sequentially, depending upon resource availability of the implementation team and inter-dependencies.

 

Keep in mind the first set of interfaces will be the most effort and longest duration because it will include all the one-time activities such as common framework development and runtime infrastructure build-out. However subsequent waves should be streamlined because those activities will have already been completed.

 

Below is an overview of suggested phases and tracks:

  • Phase 1 - Discovery and Planning
    • Track 1 - Survey, Proof of Concept, Project Planning
  • Phase 2 - Templates, Framework, and Detailed Requirements
    • Track 1 - Template and Framework Development (multiple waves)
    • Track 2 - AtomSphere Account Administration
    • Track 3 - Detailed Business Requirements (multiple waves)
  • Phase 3 - Implementation
    • Track 1 - Interface Development (multiple waves)
    • Track 2 - Runtime Infrastructure Build-Out (if hosting integrations on-premises)
    • Track 3 - Performance Testing
  • Phase 4 - Testing, Production Readiness
    • Track 1 - Functional Testing with End Users
    • Track 2 - Production Readiness
  • Phase 5 - Production Cutover
    • Track 1 - Production Deployments (multiple waves)

 

Phase 1 - Discovery and Planning

TrackActivities
Track 1 - Survey, Proof of Concept, Project Planning
  1. Conduct high level survey of existing interfaces to identify core patterns. This should not be an extensive, detailed analysis of business requirements.
    • A pattern can be a fairly loose term referring to specific business logic or technical capability.
  2. Group interfaces by pattern(s).
  3. Perform minimally-viable proof-of-concepts for each pattern (some of this may have been performed during the sales cycle).
  4. Assign, prioritize, and schedule interface groups into implementation waves.
    • Look for “low hanging fruit”: relatively simple yet meaningful interfaces that can help gain comfort with the new platform. Don’t try to migrate the most complex, critical interfaces first. Allow time for the team to acclimate to the new platform and refine operational and support procedures.
    • Consider dependencies including infrastructure, organizational, client or framework changes outside of AtomSphere, availability of clients for testing, and sensitive dates for cutovers.
    • If working with aggressive deadlines, consider de-prioritizing interfaces that are less critical or those that can be mitigated by manual processes. Involve the business team in the prioritization decisions.

 

Phase 2 - Templates, Framework, and Detailed Requirements

TrackActivities
Track 1 - Template and Framework Development (multiple waves)
  1. Develop initial template processes for patterns end-to-end
  2. Review and test template process.
    • Identify and refactor common framework components
    • Refine template and iterate
  3. Develop framework components (e.g. subprocesses) for nonfunctional requirements such as logging and error handling. Hook into existing reporting and alerting channels.
  4. process monitoring, doc tracking
Track 2 - AtomSphere Account Administration
  1. Configure AtomSphere account including:
    • Deployment environments and procedures
    • End users, roles, and access
    • Component organization and naming conventions
Track 3 - Detailed Business Requirements (multiple waves)
  1. Gather or create business requirements for the specific legacy interfaces. Depending on availability of documentation this may require essentially reverse-engineering the legacy interfaces.

 

Phase 3 - Implementation

TrackActivities
Track 1 - Interface Development (multiple waves)
  1. Develop the specific interfaces for given pattern group using the pattern templates and business requirements created in Phase 2.
Track 2 - Runtime Infrastructure Build-Out (if hosting integrations on-premises)
  1. Provision hardware servers and storage required for the local AtomSphere runtime (Atom or Molecule).
  2. Install and configure the runtime.
  3. Perform sanity testing, establish connectivity to endpoints.
  4. Configure infrastructure “plumbing”, especially when publishing web services (firewalls, load balancers, etc.).
  5. Implement runtime monitoring (e.g. JMX, log files), hook into operational alerting systems.
Track 3 - Performance Testing
  1. Establish performance/SLA requirements based on business needs.
  2. Identify and create test cases based on historical trends. Variables will include number of records, size of records, concurrent process executions, and simultaneous requests.
  3. Repeat tests on variety of runtime configurations (CPUs, memory) to dial in optimal throughput.
  4. Adjust server and AtomSphere runtime configuration accordingly.

 

Phase 4 - Testing, Production Readiness

TrackActivities
Track 1 - Functional Testing with End Users
  1. Perform end-to-end functional system testing.
  2. Coordinate testing with end users and client applications.
Track 2 - Production Readiness
  1. Business continuity planning including high availability cluster (Molecule), backup/DR failover testing.

 

Phase 5 - Production Cutover

TrackActivities
Track 1 - Production Deployments (multiple waves)
  1. Plan production cutovers at times that are least impactful to end users and client applications. Schedule around special dates such as month ends or large batch jobs.
  2. For scheduled/batch interfaces, the cutover can often be as simple as disabling the interface in the legacy system and enabling in AtomSphere. Be sure to migrate any stateful values such as last run dates, counters, etc.
  3. For web services and other event-based real time interfaces, there are a couple options. Deploy the new AtomSphere interface then:
    • If the client application must be modified to accommodate the new AtomSphere endpoint, then the AtomSphere interface will simply be ready whenever the client application is modified to point to new AtomSphere endpoint.
    • If the client application will not be modified, look to configure routing rules in the load balancer to route traffic to the new AtomSphere endpoint. You will need to consider the granularity of the API design/resources if doing a partial release. Often this will involve inspecting and rewriting the target URL based on pattern matching rules.

 

General Implementation Tips

  • Document patterns and framework components in a common place, such as an internal wiki.
  • Establish and test connectivity to application endpoints as soon as possible to work through network access issues.
  • If deploying locally, submit any internal hardware and networking provisioning requests as soon as possible.
  • Incorporate modular design elements (e.g. using subprocesses and common components) to minimize effort to make template/pattern changes in the future.
  • As the number of interfaces implementing a given pattern increases, spend more cycles refining the template. The additional upfront design and testing effort will greatly reduce the implementation effort and minimize rework.
3 people found this helpful

Attachments

    Outcomes