ETS ECT and Praxis: COBOL and JCL to Java SE 17 on AWS
How TSRI modernized the batch applications behind the world’s largest private educational testing organization, transforming 556,228 lines with only one or two lines needing a manual patch.
Executive summary
Modernizing code with advanced date-type processing queries
Educational Testing Service (ETS) is the world’s largest private educational testing and measurement organization. Each year, it develops, administers and scores more than 50 million tests, including the TOEFL®, TOEIC®, GRE® and Praxis® tests, in more than 180 countries. This requires an enormous data management capability.
ETS turned to TSRI to transform its ECT and Praxis applications and databases from IBM JCL and COBOL Batch to modern Java SE 17 on AWS, using a Spring Boot-style application to implement web services invoked by Python.
Advanced date-type queries took the bulk of the effort, yet the project still reached a 99.9996% automation rate.
01 · Challenge
Advanced date-type queries, at scale
This project was a good fit for TSRI’s capabilities and did not present any major challenges, but the legacy applications contained a large number of advanced date-type processing queries. From a transformation standpoint they were not difficult to overcome, but they demanded the bulk of the project effort, without impeding the performance of the transformation process or the result.
The result was a 99.9996% automation rate: just one or two lines of code requiring patches out of over half a million total.
Advanced date-type queries
Db2 day/time syntax to PostgreSQL
Hundreds of batch sort variations
Web services invoked by Python
02 · Approach
A model-based, rule-driven factory for transformation
TSRI ran the project using Janus Studio®, its proprietary framework and toolset for architecture-driven modernization (ADM). This model-based, rule-driven technology supports a large-scale, automated, factory process for software transformation, refactoring and documentation, refined over hundreds of projects for clients including the U.S. Department of Defense, HUD and the Canada Revenue Agency.
For ETS, TSRI applied transformation rules to a language-neutral model to modernize ECT and Praxis from COBOL to Java SE 17 without loss or change of functionality. To handle the date-type queries, the team extended the SQL transformation rules to cover day/time syntax from IBM Db2 to PostgreSQL, and made extensive additions to TSRI’s batch sort capability.
Economies of scale for future projects
The model-based approach applies source and target constraints as rules going forward, so the more it’s used, the more efficient and accurate it becomes. Future ETS projects can reuse proven conversion rules for accuracy and uniformity.
03 · Process
The five-phase Janus Studio® modernization process
Janus Studio® achieves near-100% automation through TSRI’s proprietary five-phase process.
- 01 · PHASE 1
Pre-project
Parameters, first-pass transformation, Blueprint® previews
- 02 · PHASE 2
Assessment
Parse into the Intermediate Object Model; map dependencies
- 03 · PHASE 3
Transformation
Compiling, functionally equivalent Java SE 17
- 04 · PHASE 4
Integration & testing
Resolve externals; compare legacy and modern outputs
- 05 · PHASE 5
Refactoring & final iteration
New baseline, refactoring plan, final delivery
Phase 1: Pre-project
TSRI worked with ETS to define project parameters and take in source code for a first-pass transformation, creating an Application Blueprint® and Transformation Blueprint® preview.
Phase 2: Assessment
TSRI extracted and analyzed the source code, adapting Janus Studio® to parse and transform it into the Intermediate Object Model (IOM), and gave ETS the as-is Application Blueprint® with an analysis of internal and external dependencies.
Phase 3: Transformation
TSRI transformed the ECT and Praxis code to a compiling, functional equivalent in Java SE 17 on AWS, with external references “stubbed out.”
Phase 4: Integration & testing
TSRI resolves interfaces and externals into a single instance of the application, then compares the original baseline test inputs and outputs with the modernized outputs to measure equivalency.
Phase 5: Refactoring & final iteration
TSRI takes a new baseline of production code, including updates made during the project, and applies every improvement. With ETS subject matter experts, the refactoring plan covered:
- Removing dead and redundant code
- Consolidating similar and identical code
- Rearchitecting for cloud-native targets
- Other improvements to the code base
04 · Deployment
ETS on AWS
The final codebase was delivered to ETS for integration into the target AWS environment and regression testing. This last phase allowed additional pattern-based changes while reducing code freeze; any code added since the project began can now be transformed quickly with the already-tuned rules.
- AWS Cloud and Amazon Aurora
- Amazon Cognito and IAM
- AWS Directory Service, security user pools, network ACLs
- Amazon Elastic Container Service and AWS Fargate cluster
- Amazon CloudWatch and AWS API Gateway
- AWS DataSync and AWS Storage Gateway
- ECS Fargate
- Redis caching for user interface performance
- PostgreSQL
Because of the very large number of sort cards, TSRI made extensive additions to its batch sort capability for this project.
05 · Results
Full automation, with only one or two lines to patch
Manual changes to Janus Studio® output are normally managed in a separate patch file. With a 99.9996% automation rate, only one or two lines needed patching out of 556,228 lines transformed, a relatively tiny task. Where post-automation adjustments are needed, TSRI builds them into framework code and libraries designed for reuse.
556,228 lines transformed
COBOL and JCL to Java SE 17 on AWS.
Needed a manual patch
Out of more than half a million.
Extended for date-type queries
Day/time syntax from IBM Db2 to PostgreSQL.
To completion
Including integration and regression testing on AWS.
Quality assurance with Blueprints®
QA followed TSRI’s proven process, comparing legacy results with the modernized solution. Janus Studio® generated web-based Blueprints®: the legacy system as an Application Blueprint®, and the original and transformed code side by side in an interactive Transformation Blueprint®.
Now that the ETS project is complete, TSRI can keep reusing proven conversion rules, so any code added since the project began can be transformed quickly and uniformly.