We believe that the key objectives of batch programming are, in that order : reliability, performance and maintenability.
Performance is dependant on a large set of factors, but some strong lines should be kept in mind :
- Shorten the path between the data and the data processor, whenever possible;
Reducing the network overhead for example can have a significant positive impact on global batch performance when dealing with databases.
- For critical performance needs, using the parallel processing facility can be the way to go, but generally involves some extra work to be done (smart data partitioning).
- Carefully specify the chunk size :
- A small chunk size will lower global batch performance by adding a lot of checkpoints operations;
- A very large chunk size will increase global batch performance but
- could have a significant negative impact on memory consumption;
- could involve some serious pertubations if the data are being consumed by other appliances, in case of a batch failure (since the rollback will deal with a large set of data).
- Hunt for unnecessary operations in the code; In particular, be careful with logger calls that can seriously degrade overall performance when used on a fine granularity basis.
- Follow general SQL statement performance guidelines. In particular -- a very non exclusive list ... --:
- Make sure the SQL queries are performing well, and that all databases indexes have been created;
- Pay attention to the order of restrictions (in the WHERE clause) by inspecting the execution plan,
to ensure the most discriminant filters are invoked first.
- Hunt for unnecessary outer joins.
- Only select the fields that are really needed by processing.
- Keep the I/O consumption profile low;
Regarding maintenability and reliability:
Beyond general C# programming guidelines, the main advice is : keep it simple. In particular, having some intricate steps sequences with multiple ways
within a given job make things hard to maintain and harder to test.
![[Caution]](assets/images/caution.svg) | Important |
|---|
|
This reference guide makes the assumption that you know your way around Visual Studio and creating projects and solutions.
However, to help users set up their first Summer Batch project, we wrote the Getting Started guide. We believe that
reading this tutorial should be done prior to digging into this reference documentation.
|
