All posts by chris

What Kind of Job is Right for You?

I recently came by an interesting article that examined a certain personality aspect and what it means for choice of job. That aspect was the desire to make your own rules in the workplace vs. enjoying to optimize something given. Obviously someone who does not like to work under imposed rules will be better off by starting their own business rather than joining a large organization. So far, so good.

There is, in my view, an additional “angle” to this. Looking deeper into the optimization of existing rules, processes, policies, etc. there are two conceptually different levels to it. The first is gradual improvement, and I suspect this what most people think of. It basically means that you do your job and once you have enough experience and the appropriate mindset, you start seeing opportunities where things can be done better.

But where it really starts to get interesting is when we talk about radical changes. A current example is the switch of large parts of the software industry to a subscription-based licensing model. This has implications for an enormous variety of aspects (and their KPIs for that matter). A few examples would be

  • Sales: Strategy, staff training
  • Finance: Treasury, reporting, investor relations
  • R&D: Release planning, product quality
  • Legal: Contracts (number, variance, …)

The problem with the examples above, is that it is a purely functional view. The latter, by definition, creates challenges for processes that cross functional boundaries. And I daresay that most core processes are not confined to a single department these days.

So, coming back to job types, what do I do if I have a really fundamental understanding of the situation and its challenges as well as great ideas how to handle things, but do not like corporate politics? I am certain that those three factors (truly understand the problem, have a sound solution, no politics) are a fairly typical combination.

The reason lies in what kind of jobs people with a certain personality typically choose. The people who are interested in deeply understanding things are usually more on the reserved, and often introverted, side of the spectrum. Also, they typically hate office politics and just want to do a great job on the “content level”.

Coming to a (preliminary?) conclusion this means that in most organizations people have three options and they come down to old “love it, change it, or leave it”. I can accept office politics as a by-product of a really interesting job and learn to live with them. Most people, I guess, will stay silent and just do their job but more or less quit internally. And a few brave souls will look out for something new.

My Problem with Conventional Wisdom

More and more I have realized over the last few years that I have a really big problem with many positions that are considered “conventional wisdom” or best-practice. And this goes for a wide variety of topics: software development (which I consider my main competency), technology as a whole, management, career planning, strategy, etc.

In the areas where I have developed a high level of expertise it has been clear for the longest time, because there I could easily see what conventional wisdom usually is: An oversimplified version of the true matter, that anybody can talk about assertively after having spent one or two hours with it. It is a bit like at university when after a 90 minute lecture some people really think they now understand what normalization in the context of the relational data model means.

But in reality the conventional wisdom is usually the lowest common denominator, and that only for a specific context. If you understand this and take it as a starting point for a real discussion, that is fine. But never fool yourself and think you have understood the topic. Unfortunately, this is a somewhat unpopular position and many people are offended by the mere suggestion that something is more complex than they thought so far.

What makes this a problem is when you combine it with another conventional wisdom: “Any decision is better than no decision.” Really? I think it is downright stupid. Of course, one should avoid what is called analysis paralysis and define a threshold for when a decision needs to be made. But all too often it is just an excuse to make a decision without having the slightest clue what its consequences are. Or to quote Herodotus (484-452 BC): “Quidquid agis prudenter agas et respice finem” – whatever you do, do it wisely and consider the end.

Back to the actual topic: Assuming that conventional wisdom comes into being roughly the same way for all subjects, this also means that you should stop accepting it for things you don’t know much about. Think of it as a news clip-version of something really complex. It is probably not completely wrong, but certainly an extremely limited version of the true matter. If you want to understand things, you need to dig deeper.

When looking at this from a strategic point of view, conventional wisdom in many cases means the same as commodity. If you are a product company (and that includes software) going for what everybody else does, will not help. Rather, you should spend some time thinking what you want to do really well. Believe me, it is not convincing when you interview four vendors and they all tell you more or less the same. All you know after such talks is that none of them really has a grasp of the business they are in.

Conventional wisdom will allow you to play it safe in terms of office politics, but not in terms of market success. Is that what you want?

Performance and Thread Count

I have seen many mails and forum posts, which sort-of imply that increasing the number of threads working on a task will speed things up overall. This is a dangerous path to follow in many cases, because it has the major drawback of adding overhead.

In some cases there is a range where the increase of parallelism indeed improves the situation. But that is only because of very particular performance characteristics. Basically what needs to fit is the ration of in-system execution time vs. wait time for other systems. And the critical “side” condition for the other systems is that there is no resource congestion/conflict there: I.e. if you decide to add more load (by increasing the number of threads on your side) to an external database that is already maxed out, how do you expect your code to run faster?

Lastly, there is neither a one-size-fits-all number of threads to recommend, nor can one always say that adding threads will make things better at all. The opposite can happen quite easily and it always needs proper testing with full load. Putting the latter onto the system is harder than most people think, unfortunately. But the benefit will very often be a profoundly improved understanding about the application and especially its TRUE bottlenecks. Parallelism will almost always yield results very different from a single-threaded execution (e.g. when run from your IDE).

Let me close by saying that in a surprisingly high number of scenarios, and again there is a number of pre-conditions associated, a single-threaded execution will give the highest performance (which we have not defined as a term at all here, by the way). This is because the overhead mentioned at the beginning kicks in. If you manage to have your code run such that you reduce or even eliminate cache misses on the CPU, you can see improvements by a factor of 10,000+ (and that is really not percent but a factor). But in general the effort to achieve that outweighs the usefulness; unless we talk about things like high-frequency trading of course.

New Notebook

After more than seven years my beloved MacBook Pro 15″ (late 2011) has died the GPU death. Before that I have had at least three replacement motherboards, the first after less than 24 hours of owning the thing. All these replacements were free-of-charge and without any hassle. So no complaints from that end. But given Apple’s abysmal product strategy in recent years, and particularly the butterfly keyboard, it was no option to buy a MacBook again.

To be clear on the keyboard topic: My primary objection against the butterfly keyboard is not its reliability, or lack thereof. For me it is simply a very bad keyboard with its short-travel keys. Yes, I am lucky to own an original IBM Model M and that sets a very high bar. But there are quite a few people I know who share that verdict.

Another factor not in favor of Apple was its pricing strategy and the fact that repair is next to impossible. Why would I want to spend enormous amounts of money (approximately EUR 3.500 for what I want) on a machine that cannot be repaired, upgraded, and is thermally constrained anyway? And quite honestly, with all the systematic product issue that existed for more than a decade, why would I believe this has changed?

I still own a Mac Mini from 2011 but it is months that I last used it. And after a number of years when I was really(!) happy with MacOS, I am finally departing from the latter. Overall, I got the impression that the management folks of Apple are going though the “harvesting” phase, where bean-counters have taken the reign after the founder(s) were gone. The same has happened to Hewlett-Packard, which I have had the privilege to work for many years ago. They, as so many others, had the same problem: People at the helm that do not really understand the business, but just apply methods they learned at business school. And for a number of years that works. Until it does not anymore …

So I ended up in the Windows camp, again. While Linux is my OS of choice on the back-end/server side, it has been failing to convince me for the desktop for the last 24 years. Yes, I started using Linux in 1995. And for special purposes I have used it on the desktop, usually via X11, where it made sense to me. In the 1990s that was technical development as well as authoring papers for university with LaTeX. And while a recent attempt, using KDE Neon, showed great improvements, there are still too many things that simply work better for me in Windows.

As to the actual notebook, I was pointed to the Razer Blade 15 (2018 base model) from various YouTube videos. It is a great machine and I had gotten a pretty good deal on Amazon for it. Everything was great – but the keyboard. The latter is something you can find mentioned in a number of reviews, but I must admit that I had underestimated it greatly. This is really sad, because otherwise it would have been the perfect machine for me. So where did I end up instead as keyboard afficionado?

Yes, a Thinkpad T model. What else! More precisely a Thinkpad T490 with a full-HD display, 16 GB of RAM and a 1 TB SSD. This blog post is actually the first one written on the T490 and the keyboard is an absolute treat. What drove IBM to sell this part of the business to Lenovo? Oh yes, the harvesting phase ;-).

webMethods Integration Server: How to Structure Packages

For the webMethods Integration Server (IS) packages are “containers” that bundle together various assets and get deployed as sort-of atomic units; this makes them more or less comparable to web apps in a servlet container.  So if you are given a certain task to solve, you need to figure out a clever way to organize the various parts of the overall implementation. Unless we talk about something trivial and short-lived (ever seen?), this is more complicated than it might appear at a first glance.

I did my first project on Integration Server 4.6 in early 2002 and can now openly confess, that the result, while working fine from a business perspective, was not something I would still be proud of today. To a small extent, the hardware equipment can be blamed. I was working with a notebook that had 256 MB of RAM, a single-core 800 MHz mobile Pentium III, a 20 GB hard disk with a whopping 4200 RPM, and a 14 inch display with a 1024×768 resolution.

You may now ask, how this can affect code quality. Most important in my case was performance. If pressing the save button literally sends the machine into a 2+ minutes period of frantic disk activity, you think more than twice before doing this. (Eventually, I got a memory upgrade to 512 MB.) And this is only the tip of the iceberg. In summary, refactoring is very hard if your concentration is constantly drawn away, because you have to wait for your machine to be ready again. Luckily we are past those issues today.

For the sake of this article let’s work with the following scenario, which is close to something I worked on about two years ago:

The business requirement is to automatically perform modifications to certain sales opportunities in the CRM system and report the result to the account manager.

Of course you can put everything in one package. There will be no need to think about dependency management and much less work for setting up the DEV environment, the CI server, deployment scripts (Chef in my case), etc. And in fact this is how I started. Pretty soon I moved some really generic utility services like data type conversion or date calculation into a separate utilities package. But other than that things stayed quite monolithic.

It didn’t last long, though. When I was half-way through with the initial implementation, the next, more or less separate business scenario that involved the CRM system showed up. Obviously, there was no point in having connection details, services, and other configurations related to the latter multiple times. So I moved all CRM-related stuff to yet another package.

To be clear: I was not taken by surprise when this happened. Instead it was a deliberate decision to be very careful with upfront framework building. This has been one of my personal rules for a while and it has the great advantage of being efficient as well as effective. The efficiency comes from the fact that no time is spent on things that later turn out not to be needed a second time. More important, however, is the effectiveness. It means that the outcome is useful. If you ever came across an API that is cumbersome to use, you know what I’m talking about.

Coming to an end, here is the abstract description of how I structure my IS packages. There are certainly other ways, but this works really well for me and not once in many years has caused issues or created that gut-feeling, which tells you that something is not right, although you cannot say what it is.

  • Business logic: Here is where specifics of the business side are handled. What are the criteria to select an opportunity for processing? What changes shall be applied? Technical details of the CRM system are not in scope here.
  • System details: How to perform a query against the CRM system using the criteria coming from the business logic level above? What are the connection details for a given environment (DEV, TEST, PROD)? Not in scope is e.g. the implementation of auditing.
  • Common topics: How to send notifications (e.g. where is your mail server)? Auditing, dealing with data structures, date manipulation, logging, etc.

In terms of naming conventions I suggest the following

<COMPANY_INITIALS>_<TYPE>_<DESCRIPTION>

Here are a few examples for the famous Acme Corp.

  • AC_APP_CrmOppCheck   : Application that performs checks on sales opportunities
  • AC_CMN_Audit                     : Auditing
  • AC_CMN_Util                         : Utilities
  • AC_SYS_CRM                         : Connectivity and services for CRM system
  • AC_SYS_ERP                          : Connectivity and services for ERP system

If you have questions, please do not hesitate to ask in the comments.