Embrace Modular Technology & Agile Process to Deliver Business Impact (Section 3)

Modularization for Productivity Gains 

Modularization may be described as breaking down a system into smaller component pieces that are expected to carry out independent operations.

Modularization is a technique to divide a software system into multiple discrete and independent modules, which are expected to be capable of carrying out task(s) independently. -Tutorialpoint

 

Identify the Purpose

Modularity can improve various technical situations, and in its different ways may suit one goal over another. If the modularity is to allow different versions of a framework or differing frameworks, the underlying purpose of their co-existence will help to rationalize where boundaries are split. If differing domain models make sense to divide, then again, a functional division may be identified which is at odds with layering. On the other hand, if the modularity is driven purely by a wish to remain scalable and to reduce the coupling between pieces to allow easier upgrades, then infrastructural boundaries will likely be indicated.

In any event, the equation keeps the decision of modularizing in the language of the big picture of productivity and planning for efficiencies. In turn, this helps modularity claim its place in business level discussions and decision-making, and not be thought of as a technical subtlety.

Types of Modularization

Language Level Modularization & Patterns

Modularization has helped drive programming language design since early generations of language. The ability to break up methods into smaller units, and most significantly the developments of object orientation and encapsulation behind interfaces, has helped to isolate specifics in ways that allow for multiple implementations and later replacements to fit in neatly. Type systems, which may complicate the modularization of builds, give this isolation a semantically rich and meaningful contract. Microsoft’s .NET Common Language Runtime allowed this strategy to be shared between multiple languages, as did later Java Virtual Machine languages which took the ecosystem wider than just Java itself.

Beyond the languages, patterns emerged taking the benefits of modularity further, from SOLID principles helping build stronger object orientation, to practices of layered architecture which kept data access in its own place. Already these concerns tie in with some of the business goals of modularization. Strong encapsulation and layering allow for platforms to build up with a degree of isolation from feature-supporting code which relies on them. Translated to Agile processes, this can mean the difference between an Agile team staying safely inside its lanes while leveraging the best a platform can offer, and a team either having to tread cautiously because edits will inevitably affect many product areas at once, or compromising layering altogether and creating new duplicative infrastructural code in various places. In turn, compromises to layers will make the work exponentially harder when it comes to upgrading or replacing an infrastructural concern since there may not be a single incision point in the codebase.

Software build processes are also a target of focused modularity. A module depending on another module in a build should not, in turn, be required by that module, as cyclic dependencies at a minimum point toward undesirable coupling of module code. Build engineering would seem a more localized concern than other topics of this paper, but in an enterprise codebase, it is entirely common to find a compromised build where one module will not build in isolation, even though the project is laid out in a modular way. In the day-to-day work of a small, Agile sprint, this can translate to how long it takes for each single developer to run tests against changes they make. Combined with modern source control flows, with merges of many branches being a daily activity, the ability to test modular portions of code can be critical to efficiency.

Layering with Software Frameworks

Frameworks grew in importance as enterprise code increasingly leveraged more off-the-shelf code to handle functionality shared by all applications: database access, messaging, view layer logic, serialization and so on. Frameworks play a big role in productivity, and good experiences with various frameworks and languages caused a shift towards more framework-oriented development. Depending on the underlying language as well as the design and ecosystem of these, enterprise code would to a greater or lesser degree be decoupled from the framework assumptions. As with language considerations, this is where some nuances can end up playing a divisive role later.

An example is dependency injection. Since early in its history, the most popular Java framework – Spring - provided excellent support for dependency injection. At the same time, Spring was also careful to follow Java community standards such that annotations applied to a codebase to be acted on by Spring could also be understood by different frameworks should an organization choose to go a different way. Likewise, the Java community arrived at standards for persistence annotation, so that different Object Relational Mappers could be plugged in without changing application code.

Furthering the example of dependency injection is Microsoft’s .NET offering. For a long time, .NET has supported dependency injection through 3rd party tools like AutoFac, Unity, and Ninject, among others. Nonetheless, more recently with the release of .NET Core, dependency injection has become a first-class citizen no longer requiring 3rd party tools, although they are still supported. The limit of frameworks that can be integrated, scoped, and tested using dependency injection in .NET is limited only by the developer’s imagination and the project’s architectural needs. The abstractions now built into .NET make tasks such as Test-Driven Development (TDD), changes in business logic at runtime, or injecting logic through middleware easy to develop, integrate and plug-and-play using well understood best practices.

Not all frameworks are alike in this, and some software frameworks perceived as supportive of productivity may provide poorly for upgrades, demanding many changes and targeting new greenfield projects ahead of existing ones. Patterns adopted in application logic to work well with a framework may translate clumsily into alternatives if they later need dropping in. Stronger typed systems may make boundaries and encapsulations more obvious, while in some dynamically typed code, replacing libraries or pieces of infrastructure may not present their incompatibilities and complexities until the code is run.

Informed trade-offs need to be made, between maximizing the off-the-shelf efficiency of a framework, and adding enough layered code to future proof change should a dependency become legacy and unsupported. Project code complexity can be thought of as a measure of how maintainable a codebase will be, so the adoption of software frameworks, while often a practical necessity with benefits, has serious long-term implications.

Message Based | Microservices

With message-based systems and microservices, modularization entered a new phase. Enterprise Service Bus architectures connected code via growing patterns of message queues, adapters and listeners. Where this approach lived in a more prescriptive, specific sector of software development, a broader definition of microservices became commonplace across all types of server-side software development.

In place of language types, the encapsulation comes in the form of RESTful service endpoints or in the defined semantics of the messages being published to persistent queues. A particular note of caution is to be clear of the difference between data and the interface, as complicated query parameters may cover validly variable shapes of data or equally may hide a lack of clear API definition. The modularity gains from separated services can be skin deep if detailed entrails of services are leaked to other services and opportunities missed for stricter contracts.

One appeal of these systems is to reduce complexity within each service. Another benefit is to allow for greater code differences between the various parts. For domains involving specific code approaches such as machine learning, this polyglot approach can have appeal. Even so, where the justifications are weak, deliberate fragmentation of a codebase can make it harder to understand. Perhaps less glamorously, separation of code between services allows different frameworks, and just as crucially, different versions of frameworks, to exist at once in the same codebase. This will be discussed further in the Agile and Productivity section.

User Interfaces | Separate Module or Modules

User Interfaces may best be thought of as themselves being a good target of modularity. Commonalities between the modeled concepts may be strong between the back end and front end. Notwithstanding, this reliance is a big risk for both sides of the equation. User experience has unique variables and considerations, along with changes in expectation and best practice, and must be free to evolve outside the rest of the system.

Microservices on the back end may be split by user feature areas, by more generic layering of infrastructure needs, or a combination of the two. Front end code does not need to follow the same chosen separations. A single area of a single page application may communicate data with various parts of the system whole; whether they do so via one or multiple entry-points is a choice of implementation with nuanced implications.

Sometimes the choice of service boundaries will point towards data bottlenecks which will affect user experience, and consideration will need to be given to whether the obstruction will be soluble via some relaxation of the need for real-time reflection of data.

Outside of this, recognize that the UI will likely need refactoring and restructuring as users of the application want different things, when UI or device trends change, or when frameworks evolve in unexpected ways. The ability of a system to have a new front end or front ends talking to the existing RESTful APIs represents perhaps the highest ROI of modularity in a user-facing system.

Modularity may serve an important role in separating parts of the front end itself. Redux is an example of layering working effectively in the front end, giving data its own space orthogonal to page logic and layout code. Higher order components or complex services can be examples of separating cross cutting concerns in JavaScript code in a way which helps a clean approach. Nevertheless, the front end tends to be a domain where code organization and clean sharing of functionality is still more challenging, and the benefits and costs of approaches must be weighed carefully.