Book a demo

ETS ECT and Praxis: COBOL and JCL to Java SE 17 on AWS

How TSRI modernized the batch applications behind the world’s largest private educational testing organization, transforming 556,228 lines with only one or two lines needing a manual patch.

99.9996% automation rate
Java SE 17 on AWS
11 months
CLIENTEducational Testing Service
SOFTWAREECT and Praxis applications
LANGUAGE PAIRINGCOBOL & JCL → Java SE 17
TIME TO COMPLETION11 months

Executive summary

Modernizing code with advanced date-type processing queries

Educational Testing Service (ETS) is the world’s largest private educational testing and measurement organization. Each year, it develops, administers and scores more than 50 million tests, including the TOEFL®, TOEIC®, GRE® and Praxis® tests, in more than 180 countries. This requires an enormous data management capability.

ETS turned to TSRI to transform its ECT and Praxis applications and databases from IBM JCL and COBOL Batch to modern Java SE 17 on AWS, using a Spring Boot-style application to implement web services invoked by Python.

Advanced date-type queries took the bulk of the effort, yet the project still reached a 99.9996% automation rate.

01 · Challenge

Advanced date-type queries, at scale

This project was a good fit for TSRI’s capabilities and did not present any major challenges, but the legacy applications contained a large number of advanced date-type processing queries. From a transformation standpoint they were not difficult to overcome, but they demanded the bulk of the project effort, without impeding the performance of the transformation process or the result.

The result was a 99.9996% automation rate: just one or two lines of code requiring patches out of over half a million total.

Advanced date-type queries

Db2 day/time syntax to PostgreSQL

Hundreds of batch sort variations

Web services invoked by Python

02 · Approach

A model-based, rule-driven factory for transformation

TSRI ran the project using Janus Studio®, its proprietary framework and toolset for architecture-driven modernization (ADM). This model-based, rule-driven technology supports a large-scale, automated, factory process for software transformation, refactoring and documentation, refined over hundreds of projects for clients including the U.S. Department of Defense, HUD and the Canada Revenue Agency.

For ETS, TSRI applied transformation rules to a language-neutral model to modernize ECT and Praxis from COBOL to Java SE 17 without loss or change of functionality. To handle the date-type queries, the team extended the SQL transformation rules to cover day/time syntax from IBM Db2 to PostgreSQL, and made extensive additions to TSRI’s batch sort capability.

Economies of scale for future projects

The model-based approach applies source and target constraints as rules going forward, so the more it’s used, the more efficient and accurate it becomes. Future ETS projects can reuse proven conversion rules for accuracy and uniformity.

03 · Process

The five-phase Janus Studio® modernization process

Janus Studio® achieves near-100% automation through TSRI’s proprietary five-phase process.

  1. 01 · PHASE 1

    Pre-project

    Parameters, first-pass transformation, Blueprint® previews

  2. 02 · PHASE 2

    Assessment

    Parse into the Intermediate Object Model; map dependencies

  3. 03 · PHASE 3

    Transformation

    Compiling, functionally equivalent Java SE 17

  4. 04 · PHASE 4

    Integration & testing

    Resolve externals; compare legacy and modern outputs

  5. 05 · PHASE 5

    Refactoring & final iteration

    New baseline, refactoring plan, final delivery

01

Phase 1: Pre-project

TSRI worked with ETS to define project parameters and take in source code for a first-pass transformation, creating an Application Blueprint® and Transformation Blueprint® preview.

02

Phase 2: Assessment

TSRI extracted and analyzed the source code, adapting Janus Studio® to parse and transform it into the Intermediate Object Model (IOM), and gave ETS the as-is Application Blueprint® with an analysis of internal and external dependencies.

03

Phase 3: Transformation

TSRI transformed the ECT and Praxis code to a compiling, functional equivalent in Java SE 17 on AWS, with external references “stubbed out.”

04

Phase 4: Integration & testing

TSRI resolves interfaces and externals into a single instance of the application, then compares the original baseline test inputs and outputs with the modernized outputs to measure equivalency.

05

Phase 5: Refactoring & final iteration

TSRI takes a new baseline of production code, including updates made during the project, and applies every improvement. With ETS subject matter experts, the refactoring plan covered:

  • Removing dead and redundant code
  • Consolidating similar and identical code
  • Rearchitecting for cloud-native targets
  • Other improvements to the code base

04 · Deployment

ETS on AWS

The final codebase was delivered to ETS for integration into the target AWS environment and regression testing. This last phase allowed additional pattern-based changes while reducing code freeze; any code added since the project began can now be transformed quickly with the already-tuned rules.

TARGET ARCHITECTURE · AWSSIMPLIFIED
SCHEDULING & ENTRY
Active Batch schedulerAWS API GatewayAmazon SQS
BATCH SERVICES · ECS
Async-mode batch servicesSync-mode batch servicesTSRI Job Status ManagerBatch job Python script
TSRI FRAMEWORK
TSRI DAOBatch Externals LibraryBatch Externals ServiceTSRI Java Framework Library
DATA
PostgreSQL / Amazon AuroraAWS DMSAWS DataSyncAWS Storage Gateway
PLATFORM & OPS
Container registryECS task definitionsAWS FargateAmazon CloudWatch
Simplified from the original ETS AWS architecture diagram; connections between components are not shown.
AWS SERVICES USED
  • AWS Cloud and Amazon Aurora
  • Amazon Cognito and IAM
  • AWS Directory Service, security user pools, network ACLs
  • Amazon Elastic Container Service and AWS Fargate cluster
  • Amazon CloudWatch and AWS API Gateway
  • AWS DataSync and AWS Storage Gateway
THIRD-PARTY AND OPEN SOURCE
  • ECS Fargate
  • Redis caching for user interface performance
  • PostgreSQL

Because of the very large number of sort cards, TSRI made extensive additions to its batch sort capability for this project.

05 · Results

Full automation, with only one or two lines to patch

Manual changes to Janus Studio® output are normally managed in a separate patch file. With a 99.9996% automation rate, only one or two lines needed patching out of 556,228 lines transformed, a relatively tiny task. Where post-automation adjustments are needed, TSRI builds them into framework code and libraries designed for reuse.

99.9996%automated

556,228 lines transformed

COBOL and JCL to Java SE 17 on AWS.

≤2lines

Needed a manual patch

Out of more than half a million.

SQLrules

Extended for date-type queries

Day/time syntax from IBM Db2 to PostgreSQL.

11months

To completion

Including integration and regression testing on AWS.

Quality assurance with Blueprints®

QA followed TSRI’s proven process, comparing legacy results with the modernized solution. Janus Studio® generated web-based Blueprints®: the legacy system as an Application Blueprint®, and the original and transformed code side by side in an interactive Transformation Blueprint®.

Now that the ETS project is complete, TSRI can keep reusing proven conversion rules, so any code added since the project began can be transformed quickly and uniformly.

Reusable, proven rulesFewer manual errorsFaster future projectsConsistent modernized code

Get modern fast.
Stay modern forever.

See Juno in action!