MUMPS Transformation

VA CIO 2004-2006

The Veterans Health Administration (VHA) awarded a contract to Science Applications International Corporation (SAIC) and TSRI to develop and demonstrate an automated approach to the modernization of 300,000 lines of legacy MUMPS application language into J2EE Java coupled to an Oracle 9i database. This pilot project demonstrated the feasibility of a fully automated conversion of MUMPS to Java and achieved transformation encompassing nearly all MUMPS language features. The technical strategy sought and delivered clear separation between application logic business rules, data handling presentation elements, and web-enabled front-end/user screens.

2011/2012

CRADA RESEARCH SUMMARY

MHS PEO

MHS CTO

Data Analysis of 21 Million Lines of OpenVista Meta-Data. During the remainder of Year 1 of the CRADA TSRI accomplished the following objectives:

1. Automatically Extracted Medical Module from World Vista Code
2. Used metadata to Derive Comprehensive Medical Domain Index (MDI) for Medical Domain Module
3. Attached MDI to a Transformation Blueprint^® Index of the Medical Domain Module
4. Automatically derived user metadata analysis to enhance EGL code for the WorldVistA Medical Module
5. Every MD term or MD data schema links to all uses of every part of the code that uses it.
6. MD Schema link to UML Program and UML Data Classes in fully automatically derived Medical Domain Package Index
7. Automatically replaced cryptic numeric references with meaningful medical domain names.
8. Derived Medical Domain Schema and data structures for the Dental Module
9. Derived fully qualified fields
10. Automatically derived enhanced UML designs with side-by-side views of MUMPS code.
11. MD Schema link to UML Program and UML Data Classes in fully automatically derived MD Package Index
12. Automatically Derived Meta-Data Enhanced UML
13. Medical Domain Classes * Medical Domain Call-Tree * Medical Domain Data-Flow * Medical Domain Control Flow * Medical Domain Business Rules * Medical Domain Cause-Effect Models * Medical Domain State Machine Model
14. Complete UML Model for Medical Module of interest
15. Used MDI Index to identify all code related to a specific Medical Domain of interest names (e.g. Rationalized Data Names throughout the EHR code) for the Dental Module

This research effort tested the ability of TSRI to execute technologies, methods, and processes to result in a 100% automated conversion of the Military Health Systems (MHS) AHLTA TC2 system that is in current use in the theater, from several million lines of source code implemented in the legacy programming language MUMPS, into functionally equivalent EGL.

EGL was hypothesized to provide sustained maintenance advantages over MUMPS. MUMPS is a relatively cryptic language with terse acronyms that obscure the meaning of the code. It requires specialized MUMPS developers that are in shorter supply compared with programmers versed in mainstream modern languages such as C, C++, Java, and C#. Furthermore, mainstream programmers find MUMPS difficult to read and understand. The re-expression of MUMPS in a modern language such as EGL seemed to be, and proved to be, a rational strategy to reduce maintenance costs and mitigate technical risk.

The deadline to achieve blueprinting for the above was June of 2011. TSRI commenced the research plan outlined in March of 2011, completed delivery of a complete TC2 MUMPS to EGL Transformation Blueprint^® in May, and conducted review for the MHS CTO in June 2011.

3-months after initiation of the CRADA, TSRI achieved success with the AHLTA blueprint objectives. Specifically, TSRI generated an AHLTA TC2 MUMPS to EGL Transformation Blueprint™: A detailed presentation of the before and after code, baseline target system design, and architecture of the modernized AHLTA MUMPS code expressed in EGL. This was presented in an interactive UML design in supported of decision making regarding the CRADA hypothesis. The blueprint was presented in four major views:

• As-Is and To-Be/Both Source Code View: (the side-by-side a composite ’Both’ view of the As Is AHLTA TC2 MUMPS & To Be EGL with line-level hyperlinks between the hypertext of the source and target code views for the ‘before’ and ‘after’ AHLTA TC2 System
• AHLTA TC2 As-Is/To-Be Blueprint Hypermodel: presentation expressed as: Hyperlinked SVG Diagram: Structure Charts, Control-Flow Graphs (a.k.a. Action Diagrams), Data-Flow Diagrams, State Machine Graphs, State Transition Tables, Cause-Effect Graphs, Class Diagrams, and Data Element Tables linked to hypertext of the EGL code
• Blueprint Metric Indices: Cyclomatic Complexity Index; Call Fan-In, Fan-Out Index; Class Subclass and Superclasses Indices; State, Rule, and Statement Complexity Indices; Dead Code Indices, Duplicate Code Indices; Un-referenced Variables Index; System Package Deployment Index, System Data Element Tables hyperlinked to the EGL hypertext code; and
• Multiple Composite Views: (functional analysis-level design modeling perspectives as depicted by the following leading modeling methodologies): MDA (i.e. UML) View – Model Driven Architecture (Unified Modeling Language),SASDM View – Structured Analysis System Design Methodology View, OOA/OOD View – Object Oriented Analysis / Object Oriented Design View, BRM View – Business Rule Modeling Methodology View, SMM View – State Machine Modeling View.

After achieving the above objectives early in the CRADA, TSRI and MHS, by mutual agreement, expanded the objectives of the CRADA to focus on utilizing metadata to improve the maintainability of the transformed code by deriving a Data Dictionary (Medical Domain Index) and mapping all utilization of MUMPS global data to the point of use within the derived code.

2012/2013

Military Health System (MHS) Transition Application Plan Support (TAPS) ID09120057

CHCS M to Java Translation Proof of Concept Analysis

TSRI achieved its objectives of demonstrating a 100% automated transformation from MUMPS to functionally-equivalent Java for a subset of CHCS MUMPS code using the strategy developed for the 2005 VA Volunteer Timekeeping system.

1. Data was collected to facilitate estimation of the effort required for testing a larger code base. 33% testing productivity increase was measured when comparing the first half to the second half of the testing activity.
2. TSRI achieved 100% automated transformation to compiling Java for just over 1 MSLOC of CHCS related MUMPS code.
3. Effort to modify JANUS to generate compiling Java was not a significant portion of this project and is not expected to play a significant role in future projects which use this strategy.
4. The SQL Server RDBMS was replaced by an InterSystems Globals database and the data access tier was extended to integrate with the new database. As all the details of the data persistence are isolated in the data access tier, this was possible without a change to the generated Java code. This alternate configuration demonstrated ~12x higher performance during the data migration activities and substantially higher performance during the radiology use case execution, though the change in use-case performance was not quantified.

Director DOD/VA IPO

MHS PEO

By mutual agreement TSRI and MHS extended the CRADA for a second year and achieved the following objectives during the 2012/2013 year of the project:

1. Create metadata enhanced Transformation Blueprint^® preview for all of TC2 using TC2 Meta Data Significant Success Achieved: This was partially accomplished with about 52% of meta-data achieving resolution for TC2 vs 41% for WV.
2. Combine the MHS TC2 and VHA VistA systems (World Vista is the open source versions of the VHA VistA system) during the automated transformation process. Significant Success Achieved: Both TC2 and WV were combined and transformed. Runtime to transform and apply code redundancy refactoring to both systems was approximately 12 hours for 47,000 files and 4.2 Million lines of code.
3. Achieve high-levels of code size reduction during consolidation of TC2 with VistA during the transformation of the two systems. Significant Success Achieved: Substantial size reduction was demonstrated on both systems.
4. Achieve high levels of algorithm optimization and parallelism and during modernization process to demonstrate scalability. Significant Success Achieved: The run-time for code transformation and refactor for two systems was reduced from would never finish to under 12 hours through algorithm optimization.
5. Create a combined Blueprint displaying the modernized code, design and architecture models Partial Success Achieved: This was demonstrated for up to 10,000 files from both systems and eventually expanded to two complete systems.
6. Apply design-pattern oriented refactoring to extracted Utility Modules. Partial Success Achieved: We applied global brute-force block-level refactoring (redundancy reduction) and DAO patterns for data accessor method encapsulation. The benefit of this approach was that hundreds of thousands of methods were formed to encapsulate code that was reusable; however, the downside of this approach was that stronger techniques were needed to derive meaningful method names for automatically derived code units. TSRI has since made progress towards semi-automated business domain model and business rule model derivation. We believe this approach to automated business rule extraction can be leveraged to automatically assign meaningful names to automatically derived methods.
7. Apply design-pattern oriented refactoring to extract common base classes. Partial Success Achieved: We applied global brute-force common base class refactoring successfully. This approach is optimal when refactoring into an object-oriented language. We did not, however, generate metrics to measure the degree of effectiveness of this design-pattern oriented refactoring. Our approach sought to maximize the degree to which common code could be consolidated into base classes and employed multiple inheritance techniques to achieve this goal. Multiple inheritance is available in languages such as C++; but, in languages such as C# or Java it is not permitted and consolidation into base classes must observe single inheritance. Single inheritance base class formation requires polynomial time, and while single inheritance is achievable by adjusting the consolidation algorithm parameters, the consolidation algorithm has not yet been optimized to minimize runtime. To address the needs of this RFI, the consolidation algorithm would be adapted to handle consolidation of multiple systems, but would initially be limited to the consolidation of duplicate code and data.
8. Apply DAO-pattern oriented refactoring during transformation to support an automated interface with ORM technologies such as hibernate and coherence. Partial Success Achieved: We applied DAO pattern-refactoring to create DAO classes for derived data structures. During this early period of our work with and research regarding MUMPS, only 52% of metadata was resolved to meaningful names. We subsequently increased our coverage to above 98% when applied to TC2 and VistA. At the time, no mapping yet existed for Hibernate or Coherence to support the definition of an object-relational mapping. Subsequent work achieved insertion of meaningful names into Java code which referenced Java data access object (DAO) classes throughout entire MUMPS VistA system instances.
9. Construct DAO layer that encapsulates field references to data structures derived from TC2 and VistA meta data. Partial Success Achieved: DAO pattern encapsulators were created for data access method for data access patterns. Limitations: Only 52% of metadata was being resolved.
10. Demonstrate step-wise refactoring using the approach applied to Eurocat (i.e. user defines a refactoring specification via an external interface. JANUS executes the specification). Partial Success Achieved: We activated this capability and applied it to small examples. While we demonstrated a capability to execute step-wise refactoring on MUMPS during transformation, the construction of refactoring plans is greatly assisted by 3^rd-party visualization tools such as Klocwork , which allow a user to visually inspect and assess the design and quality of coupling. We currently support a general interface for the import of refactoring plans from external sources such as Klocwork.
11. Automatically derive medical modules (à la the DENT module) for VISTA and TC2 (i.e. Define a heuristic for grouping modules together based upon their functional and structural cohesion). Partial Success Achieved: We built the technology for fully automated derivation (mining of modules) from any set of designated data structures. However, some expertise is  required to assist with module derivation.

PEO DHCS

CTO MHS

Director of TATRC

By mutual agreement TSRI and MHS extended the CRADA for a third year and pursued the goal of determining “the best practices of auto-converting TC-2, AHLTA-T, AHLTA, CHCS and VISTA into a unified executable code set and improving the TSRI JANUS tool set.” The key accomplishments of the third year are summarized below:

1. Achievement of what amounted to ‘push-button’ conversion. The translated code ran identically to the original code and all user interactions, PF keys and other novel behaviors were preserved.
2. No human intervention was required after conversion of code. MUMPS modules could be directly replaced with the JSE/JEE modules generated by the automated transformation process.
3. No operator retraining was required for transformation. Generated code was fully deployable in data centers as x86 SaaS hosted applications running on conventional PC web browsers or handheld devices identically to the original MUMPS.
4. The original code and modernized code proved to be interoperable and could be run in parallel. The performance of the transformed code proved superior to the source MUMPS code, and was elastic and scalable in cloud-based PaaS/IaaS.
5. The code transformation was demonstrated to be highly accurate. Initially, a five sigma rating level of accuracy was achieved (less than 1 functional error per 100,000 lines of code translated). This was improved to a six sigma level of accuracy (less than 1 error per 1,000,000 lines of translated code by the end of the project).
6. The transformation proved highly efficient, and successful conversion of a single system of 2.1 to 2.5 Million lines of code could be delivered in a little over an hour (1 hr 20 min)
7. The modernized application’s database performance proved to be equivalent or better than that of the original application when tested with real-world equivalent test data, but without instrumentation monitors.
8. Transformation with added computational burden of consolidation also generally proved highly efficient. Two system (e.g. VHA VistA and MHS CHCS) unification/conversion process could be completed via automated execution in three hours.

The conversion approach described above was designed to allow MUMPS programmers to continue programming in MUMPS while the VA and MHS could simultaneously deploy modernized, transformed applications into data centers and into future IT architectures. The modern code was intended to initially exist within a hybrid architecture where it would operate seamlessly alongside and with existing MUMPS code (the legacy environment). This state could intentionally continue until full transition was achieved from the legacy MUMPS architecture to the transformed modern architecture. Test evidence (as described above) largely revealed that the intended capabilities had been achieved.

This development and applied research initiative intends to further improve TSRI’s MUMPS/VistA consolidation/transformation process, and (in conjunction with OHUM Healthcare) to eventually deploy a non-MUMPS VistA system, running in contemporary coding languages, and utilizing a fully modern architecture, hosted in the cloud. It is temporarily on hold to ensure capacity for potential projects, which could influence and drive the direction of development, while making use of the research to date.

Accomplishments to date:

1. Full interoperability of OHUM’s iPad application (based on CPRS) and converted TSRI Java, indicating that OHUM’s VistA variant and TSRI’s converted Java are interchangeable at the API level
2. Development of a domain-aware Web administrative front-end
3. Full ability to comprehend FileMan schemas, including system-specific discrepancies
4. Research of MUMPS arithmetic rules, and updating of TSRI conversion’s arithmetic to match
5. Automatic DAO generation based on system-specific FileMan schemas
6. Replacement of some Kernel calls and some direct FileMan data accesses with DAO calls
7. Automatic enforcement of VA Standards and Conventions, for easier convertibility and maintainability (ongoing)
8. Improved code legibility based on VA SAC and VDL documentation (ongoing)
9. Demonstrated at OSEHRA June 2016 Summit, along with integration of a previously system-naïve medical device