Tag Archives: Refactoring

webMethods Integration Server: How to Structure Packages

Architecture, Integration, Technical Stuff, Tools, , ,

For the webMethods Integration Server (IS) packages are “containers” that bundle together various assets and get deployed as sort-of atomic units; this makes them more or less comparable to web apps in a servlet container.  So if you are given a certain task to solve, you need to figure out a clever way to organize the various parts of the overall implementation. Unless we talk about something trivial and short-lived (ever seen?), this is more complicated than it might appear at a first glance.

I did my first project on Integration Server 4.6 in early 2002 and can now openly confess, that the result, while working fine from a business perspective, was not something I would still be proud of today. To a small extent, the hardware equipment can be blamed. I was working with a notebook that had 256 MB of RAM, a single-core 800 MHz mobile Pentium III, a 20 GB hard disk with a whopping 4200 RPM, and a 14 inch display with a 1024×768 resolution.

You may now ask, how this can affect code quality. Most important in my case was performance. If pressing the save button literally sends the machine into a 2+ minutes period of frantic disk activity, you think more than twice before doing this. (Eventually, I got a memory upgrade to 512 MB.) And this is only the tip of the iceberg. In summary, refactoring is very hard if your concentration is constantly drawn away, because you have to wait for your machine to be ready again. Luckily we are past those issues today.

For the sake of this article let’s work with the following scenario, which is close to something I worked on about two years ago:

The business requirement is to automatically perform modifications to certain sales opportunities in the CRM system and report the result to the account manager.

Of course you can put everything in one package. There will be no need to think about dependency management and much less work for setting up the DEV environment, the CI server, deployment scripts (Chef in my case), etc. And in fact this is how I started. Pretty soon I moved some really generic utility services like data type conversion or date calculation into a separate utilities package. But other than that things stayed quite monolithic.

It didn’t last long, though. When I was half-way through with the initial implementation, the next, more or less separate business scenario that involved the CRM system showed up. Obviously, there was no point in having connection details, services, and other configurations related to the latter multiple times. So I moved all CRM-related stuff to yet another package.

To be clear: I was not taken by surprise when this happened. Instead it was a deliberate decision to be very careful with upfront framework building. This has been one of my personal rules for a while and it has the great advantage of being efficient as well as effective. The efficiency comes from the fact that no time is spent on things that later turn out not to be needed a second time. More important, however, is the effectiveness. It means that the outcome is useful. If you ever came across an API that is cumbersome to use, you know what I’m talking about.

Coming to an end, here is the abstract description of how I structure my IS packages. There are certainly other ways, but this works really well for me and not once in many years has caused issues or created that gut-feeling, which tells you that something is not right, although you cannot say what it is.

  • Business logic: Here is where specifics of the business side are handled. What are the criteria to select an opportunity for processing? What changes shall be applied? Technical details of the CRM system are not in scope here.
  • System details: How to perform a query against the CRM system using the criteria coming from the business logic level above? What are the connection details for a given environment (DEV, TEST, PROD)? Not in scope is e.g. the implementation of auditing.
  • Common topics: How to send notifications (e.g. where is your mail server)? Auditing, dealing with data structures, date manipulation, logging, etc.

In terms of naming conventions I suggest the following

<COMPANY_INITIALS>_<TYPE>_<DESCRIPTION>

Here are a few examples for the famous Acme Corp.

  • AC_APP_CrmOppCheck   : Application that performs checks on sales opportunities
  • AC_CMN_Audit                     : Auditing
  • AC_CMN_Util                         : Utilities
  • AC_SYS_CRM                         : Connectivity and services for CRM system
  • AC_SYS_ERP                          : Connectivity and services for ERP system

If you have questions, please do not hesitate to ask in the comments.

It’s Not Beautifying but Reduction of Complexity

Architecture, Technical Stuff, ,

I am writing this on a Saturday afternoon after having spent several hours going over Java code that is about a year old, needed only occasionally, and has not caused issues for quite a while. Still I spent personal time to improve or, shame on me, even add the Javadocs. Also, some additional error checks were added and the code structure simplified.

None of this had been asked for by anyone else. And probably quite a few people would call it a terrible waste of time, given the various new features still needed so badly. So why did I do it?

It was an experiment, a successful one  from where I stand. In the beginning I merely wanted to see in how many ways the code could be improved with sufficient time gone by after the initial writing. I ended up doing the following things:

  • Renamed class ServiceWrapper to ServiceWrapperManager, because the class is not an actual wrapper but creates and deletes them.
  • Renamed variable adapterServiceNode to node, because via a wrong user input it could point to a node object of a different type and not an adapter service. (The same was done in two other cases.)
  • Added multiple checks that would throw an IllegalArgumentException, if the user had provided wrong input. So rather than having to wonder why the creation of a wrapper had failed, I would be “shouted” at early in the process.
  • Added and improved multiple Javadoc entries. This, in turn, made me look at the code more closely where I discovered multiple things to improve or even fix.

When I look at the code now, it is cleaner, more robust, and easier to read. And the latter is the most important aspect on my case, because it reduces technical debt. So here you have your argument.

And for business people this means that I “keep my head above water level”. Or in other words: It is an absolute prerequisite for fast time-to-market that my code base is tidy and well-structured. Only then can I react quickly and change things without having to think hours what might break in other places, should I now add this or that new feature.

Revisiting Software Architecture

Architecture,

Quite recently I heard a statement similar to

“The application works, so there is no need to consider changing the architecture.”

I was a bit surprised and must admit that in this situation had no proper response for someone who obviously had a view so different from everything I believe in. But when you think about it, there is obviously a number of reasons why this statement was a bit premature. Let’s have a look at this in more detail.

There are several assumptions and implicit connotations, which in our case did not hold true. The very first is that the application actually works, and at the time that was not entirely clear. We had just gone through a rather bumpy go-live and there had not yet been a single work item processed by the system from start to finish, let alone all the edge cases covered. (We had done a sanity test with a limited set of data, but that had been executed by folks long on the project and not real end users.) So with all the issues that had surfaced during the project, nobody really knew how well the application would work in the real world.

The second assumption is that the chosen architecture is a good fit for the requirements. From a communication theory point of view this actually means “a good fit for what I understand the requirements to be”. So you could turn the statement in question around and say “You have not learned anything new about the requirements since you started the implementation?”. Because that is what it really means: I never look back and challenge my own thoughts or decisions. Rather dumb, isn’t it?

Interestingly, the statement was made in the context of a discussion about additional requirements. So there is a new situation and of course I should re-evaluate my options. It might indeed be tempting to just continue “the old way” until you really hit a wall. But if that happens you have consciously increased sunk costs. And even if you can “avoid the wall”, there is still a chance that a fresh look at things could have fostered a better result. So apart from the saved effort (and that is only the analysis, not a code change yet) you can only loose.

The next reason are difficulties with the original approach and of that there had been plenty in our case. Of course people are happy that things finally sort-of work. But the more difficulties there have been along the way, the bigger the risk that the current implementation is either fragile or still has some hidden issues.

And last but not least there are new tools that have become available in the meantime. Whether they have an architectural impact obviously depends on the specific circumstances. And it is a fine line, because there is always temptation to go for the new, cool thing. But does it provide enough added value to accept the risks that come with such a switch? Moving from a relational database to one that is graph-based, is one example that lends itself quite well to this discussion. When your use-case is about “objects” and their relationships with one another (social networks are the standard example here), the change away from a relational database is probably a serious option. If you deal with financial transactions, things look a bit different.

So in a nutshell here are the situations when you should explicitly re-evaluate your application’s architecture:

  • Improved understanding of the original requirements (e.g. after the first release has gone live)
  • New requirements
  • Difficulties faced with the initial approach
  • New alternatives available

So even if you are not such a big fan of re-factoring in the context of architecture, I could hopefully show you some reasons why it is usually the way to go.