Summer Batch - A New Season for .NET Batch Processing

The Need

The worker’s compensation board for major North American government had a problem. An early adopter of cloud computing, the organization had begun replacing its legacy systems with web-based solutions based on Microsoft’s .NET framework back in the early 2000s. It replaced its claims system in 2001 and began with its policy administration system in 2015. However, the organization was left with a missing piece; its Case Information System (CIS) was an ancient COBOL relic which remained on the IBM mainframes and had complex functionality that few individuals within the organization could still support. Further complicating the issue was the fact that it required both online and batch processing. In the mainframe era, computing power was a much more scarce resource than today. Mainframe systems were, therefore, built around batch processes, allowing maximization of scarce computing resources and postponement of transaction processing to a time when the system was less taxed by users, e.g. overnight or on weekends. Modern programming languages where not developed with these limitations in mind and, consequently, for many years lagged behind COBOL and other languages of the mainframe era in their ability to handle batch processing. They lagged behind, that is, until the arrival of Spring Batch. In 2006 Accenture collaborated with the creators of the Spring Framework to bring an open-source batch processing framework to Java. Over the next couple years, the implementation of Spring Batch at major companies worldwide proved the batch functionality of legacy systems could be replicated with a modern object-oriented programming language. Today Spring Batch is downloaded over 1.5 million times per year. While Java has chosen to support native batch processing, Microsoft has chosen to support batch processing through stored procedures within SQL Server. This solution is less versatile and taxes the processing power of the database server.

The Development of Summer Batch

When the organization approached Accenture to help them solve the problem of their CIS, Accenture had already recognized the need for Spring Batch-style functionality in .NET environments. Accenture partnered with Blu Age, a company they already trusted for .NET and Java modernization solutions, to develop the solution they needed. Blu Age, therefore, endeavored to create a Spring Batch equivalent for the C#/.NET environment using the Java batch specification, JSR 352, as a starting point. This specification is Java-oriented but the concepts behind it are universal. With an extensive catalogue of past modernization projects, Blu Age was able to incorporate additional elements that were not present in the JSR 352, such as the inclusion of a dedicated tasklet to handle EBCDIC data (see the documentation for more information).

Major Challenges

The Spring Batch implementation of the JSR 352, as part of the broader Spring framework, relies heavily on other parts of the Spring framework and offers a lot of functionality out of the box. The Microsoft environment lacks a lot of features, especially regarding inversion of control. Blu Age considered many alternative frameworks and ultimately chose Unity. This posed several technical difficulties since BA had to develop new mechanisms for the open-source Unity Application Block, which are supported by other modules of the Spring framework under Spring Batch. For instance, dependency injection was not handled by any other dependency injection framework so we had to build it for Unity. Adapting these functionalities to the Unity framework became what Blu Age architect Thierry Masson called the “spine of the project or the heart of the Summer Batch architecture”. This required extensive technical development with proxies to ensure the objects have the proper lifecycle.

Get Summer Batch here

Unity is complex to use and it took in depth analysis to come up with the right design. Building the database access layer was a much simpler challenge - almost as simple as “just switching from one technology to another”. Summer Batch uses DocBook to provide extensive and easy to use documentation. After the initial development of Summer Batch, an Accenture developer contributed additional functionality to Summer Batch. “In fact it was quite a nice addition, because it added functionality that was missing”. He added support for PowerShell. This feature can be quite powerful because most of the administration on the MS platform is through PowerShell.

The Next Step

The introduction of .NET core in 2016 posed a new challenge to the Summer Batch development team. The new framework represented a landmark commitment to open-source technology from Microsoft. However, the open-source Unity framework is not being actively developed for .NET core. The Unity components have, therefore, been replaced with StructureMap dependency injection for the new .NET core-compliant Summer Batch release, which will be arriving in the first quarter of 2019.

Summerbatch Summerbatch