I just came across another interesting statement about the size of development teams:
What makes this so interesting is its origin. It was said in 1968 during the famous NATO Conference on Software Engineering. Fifty years later I think this is still remarkably true. But why?
In my post on the size of development teams I was aiming at complexity and scope as drivers. The interesting question, though, is whether there are additional points to consider. Having worked pretty closely with some other projects in the meantime, I came across another influencing factor: availability of qualified staff.
You can look at this in several ways. The first hurdle is obviously to have enough developers with the required experience at your disposal. The second is how to allocate their capacity to the portfolio of activities you run. Or put differently: How do you deal with the shortage of know-how you face? Because, let’s be honest, there is always more demand for highly skilled developers than can be supplied. (I am having a déjà vu with my “beloved” macro economics lecture 😉 back in 1996.)
If your organization is above a certain size, chances are that overall a sufficient numbers of good people work there. Does that put you in a better position than some poor fellow working for a small company? Not really. Because your project competes for these folks with all the other work within the organization. And what is the difference compared to competition with an external market? Yes, if you have really goof connections to higher management, you can escalate things and possibly get additional people. But you will also burn bridges in the process, which is usually too high a price.
So instead you probably end up with the usual mix of folks: Very few rock stars, many middle-of-the-road folks, some promising newbies, and the occasional looser that nobody wants. Can you deliver something really great with such a mix? It will be difficult in a typical corporate setup where you have to somehow involve everybody. And this involvement of the less qualified half of people will slow down the “upper” half.
It is a very delicate subject and there are many fine lines, some from a legal perspective and many more from decency-to-others point of view. Also, the organization needs to think about tomorrow and therefore must have a “funnel” of to-be-rockstars, which need the best training they can get. And the latter is always working on a difficult project with experts and learn from them. But what few organizations do, is look at competence levels in detail and factor them in.
In other words: You can learn a lot from someone who is one, two, or perhaps three levels above you. But if someone is ten years ahead of what you currently know, the difference is just too big. You will only grasp a small fraction of what they teach you, and even that with a considerable risk of misunderstandings. And they honestly cannot understand why you are not following their great advice. Mutual frustration and dislike are usually the result.
Whether you take competence levels into account or not, considerable effort needs to be spent on non-development activities. If you have a taste for management and leading people that will be a great opportunity for you. But if your primary concern is getting something great and possibly visionary delivered, you should seriously consider a totally different approach: an underground project.
Flying under the radar can really be liberating. This must not be confused with idling around or working on some obscure pet project. It is truly about delivering what the organization needs but cannot accommodate in its own structure. Of course you should be certain that your boss will not fire you, if he or she finds out. But as you probably already guessed, driving an underground project does not mean that you are freed from politics, lobbying etc. On the contrary! You must prepare upfront quite carefully how you counter resistance or outright attacks. And yes, you are running a personal risk. But there is no reward without risk.
I cannot say how often I read in some company’s image brochure that management and expert career paths are treated equal. There must be companies on our planet where this is true, but I have yet to see it in practice. Why is that? (Unless mentioned otherwise this article is about the IT industry with a strong focus on consulting and software companies.)
In my experience the core reason is a fundamental lack of understanding from the non-technical folks that make the relevant decisions, i.e. management. They have a vague impression of techies that is often dominated by perceived social deficiencies (“They can’t look me in the eye”). While the latter is sometimes true, how can it be a measure of someone’s qualification to perform a complex technical task? And perhaps there is actually a good reason for averting my eyes. I cannot speak for others, but when I have a complicated discussion, it helps me focus all my mental capacity on the problem at hand when I look onto the table.
I would speculate that, as in so may other cases, the heart of the problem lies with mid-level management (where the details are determined) as opposed to the C-level. Taking into account the Peter principle, what does that say about middle management? Yes, this a strong simplification and it does many people injustice. I have personally met quite a few people on this level that I consider great at their job, particularly including leadership skills. But just as well did I encounter folks who cannot think out-of-the-box and blindly follow rules and processes. And it is this capability to see new ways, make realistic projections about what will (not) work, and lead by example “where no-one has gone before”.
A particularly interesting story for me was the encounter with an HR person where we talked about setting up an initiative to advance technical folks (what is often called a high-potential program). My idea was met with friendly resistance and the argument was made that contrary to management, the technical side is so much more diverse that it would be virtually impossible to have a single program. In fact this HR person seriously thought that every techie needed his or her completely individual program.
My counter argument then was that this would not be a technical program (learn programming language XYZ) but one for professional and personal development. So we talk about e.g. how to present a complex technical scenario to upper management for getting a budget approval. Or how to engage with a customer and instill confidence that a given product is the right choice to solve their business problem. And all this with being authentic and building upon the strong technical knowledge one has.
I don’t want to appear as simply bashing HR here. But the underlying problem was that assumptions had been made about what would be suitable for technical folks without ever having spoken with any of them. Would this happen the same way for management staff?
From a more abstract perspective it comes down to the fact that many decisions are made based on personal trust and relationships, rather than on processes and policies. So what is needed is that the relevant management and technical staff (on every hierarchical level) talk to each other in the spirit of equality. And here both sides fail miserably too often.
When the schedule is tight for a software development project, one of the first things non-technical people contemplate, is reducing quality and/or scope and go for an 80% solution. This can work but only under particular circumstances.
A recent discussion I had with some folks was about such a situation. A very tight deadline had been issued by senior management and multiple departments had to collaborate on a solution that spawned multiple parts of the organizations and their core IT systems. Those systems (think about something like ERP and CRM) have a multitude of connections, grown over many years without governance, but not for the data that were required now.
The people were all quite supportive and a constructive discussion took off. Not surprisingly, finding a common language took some time. Especially so because only after a while did we realize that in several cases the same term had a different meaning. In the end there was a particular detail where no obvious solution was available.
I was responsible for the technical implementation on one of the two systems involved. My counterpart on the business side and I discussed various options and and eventually he came up with the idea to just safe time by going for the 80% solution. We talked about a number of aspects and it soon became clear to me that this would actually require more time than to just “do it properly”.
This sounds, at the very least, counter-intuitive for most people. So let me illustrate my point with a non-technical equivalent. Many years ago I had to prepare for my then-boss a list with press clippings every day. It basically meant to go through a long list, that was created automatically, and remove everything he was not interested in. This was quite time-consuming and he was genuinely surprised when he found out.
His statement then was “Why do you need so much time, it is only 20 of the 200 articles that are relevant”. That was of course correct. But what he had not thought about was that I had to go through the list of all 200 articles and check some in more detail than just looking at the headline. So it did not really make a difference whether I cut the list down to 100 or 60 or 20 articles, the time required stayed more or less the same. He agreed on the point and quite soon decided there was better use for my time.
It is exactly the same with taking shortcuts during a technical implementation. The easy part is to decide that you want to cut 20% off the next release. But things get tricky when it comes to deciding which 20% to skip without any undesired side-effects. In order to be able to do that, you need to understand in depth what the consequences would be. So quite often you spend more time trying to understand potential ramifications for various scenarios, than it would take to simply do it properly right from the start.
And that is only the beginning of the problem. You need to document the results of your analysis really carefully, if you want to be sure that nothing is missed when the first code change is required. Even a seemingly minor bug fix has an increased potential to break something if the current overall solution is built on a set of assumptions. And adding enhancements or new features will be even more “fun”. Because people have to read through all the documentation and also understand it. Together this is a time-consuming task with considerable risk to still overlook something (or hit an edge-case that was not documented).
But in reality this documentation does not exist anyway because you wanted to save time in the first place. And detailed documentation is not exactly supportive here. So people throw in bits and pieces and you end up with a document that is typically even worse than most project documentation that is out there. So whoever has the pleasure to work on the code without having all the details in mind is really screwed. That includes you, by the way, because you will have forgotten most after only a few weeks.
The solution is usually a re-implementation because it is faster than to fix the broken code. Of course that re-implementation often needs additional logic to handle data migration for transactions that were completed with the 80% solution and now ruin your data quality. But hang on, that might just be 20% effort you can safe right now ….
Simon Sinek is well known for his book “Start with Why: How Great Leaders Inspire Everyone to Take Action“. Here he gives an interesting talk about why and how we trust others.
What should we do to handle the constantly changing domain and technical requirements? Patrick Kua talks about some interesting aspects on the architectural level.
It is not uncommon that an existing piece of non-trivial software gets re-written from scratch. Personally, I find this preference for a greenfield approach interesting, because it is not something I share. In fact, I believe that it is a fundamentally flawed approach in many cases, so why are people still doing this today?
Probably because they just like it more and also it is perceived, at least when things get started, as more glorious. But when you dig deeper the picture changes dramatically. Disclaimer: Of course there are situations when re-starting is the only option. But it typically involves aspects like required special hardware not being available any more.
When I started writing this post with making some notes, it all started with technical arguments. They are of course relevant, but the business side is much more critical. Just re-phrase the initial statement about re-writing to something like
Instead of gradually improving the existing software and learn along the way, we spend an enormous amount of money on something that delivers no additional value compared to what we have now. For this period, which we currently estimate to be 2 years (it is very likely to be longer), apart from very minor additions, the business will not get anything new, even if the market requires it; so long-term we risk the existence of the organization. And yes, we may actually loose some key members of staff along the way. Also, it is not certain that the new software ever works as expected. But should it really do, the market has changed so much, that the software is not very useful for doing business anyway and we can start all over again.
Is there anyone who still thinks that re-writing is generally a good idea?
Let us now change perspective and look at it from a software vendor’s point of view. Because the scenario above was written with an in-house application in mind. What is different, when we look at a company that develops and sells enterprise software? For this text the main properties of enterprise software are that it is used in large organizations to support critical business processes. How keen will customers be to bet their own future on something new, i.e. not tested? But even if they waited long enough for things to stabilize, there would be the migration effort. And if that effort comes towards them anyway, they may just as well look at the market for alternatives. So you would actively encourage your customer base to turn to the competition. Brilliant idea, right?
What becomes clear looking at things like that, is what the core value proposition of enterprise software is: investment protection. That is why we will, for the foreseeable future, continue to have mainframes with decades-old software running on them. Yes, these machines are expensive. But the alternatives are more expensive and in addition pose enormous risk.
In the context of commercial software vendors one argument for a re-write is that of additional revenue. It is often seen as easier to get a given amount of money for a new product than an improved version of something already out there. But that is the one-off view. What you typically want as a software vendor is a happy customer that pays maintenance every year and, whenever they need something new, first turns to you rather than the competition for options. Also, such a happy customer is by far the best marketing you can get. It may not look as sexy as getting new customers all the time, but it certainly drives the financial bottom line.
Switching over to the technical side, there are a few arguments that are typically made in favor of a restart. My favorite is the better architecture of the new solution, which will be the basis for future flexibility, growth, etc. I believe that most people are sincere here and think they can come up with something better. But the truth is that unless someone has done something similar successfully in the past, there is a big chance that the effort is hugely underestimated. Yes, technology has advanced and we have better languages and frameworks. But on the other hand the requirements have also grown dramatically. Think about high availability, scalability, performance and all the others. Even more important, though, is the business side. With something brand new people will have greatly increased expectations. So giving them something like-for-like will probably not be seen as success.
The not-invented-here syndrome is also relevant in this context and particularly with more junior teams. I have seen a case when an established framework used in more than 9,000 business-critical (i.e. direct impact on revenue) installations was dismissed in favor of something the team wanted to develop themselves. And I can tell you that the latter was a really bad implementation. Whether it was a misguided sense of pride or a fundamental lack of knowledge I cannot say. But while certainly being the most extreme incarnation I have seen so far, it was certainly not the only one.
So far my thoughts on the subject. Please let me know what you think about this topic.
Two years after reading Eliyahu M. Goldratt‘s famous book “The Goal” for the first time, I had a second go at it. It proved, again, to be an entertaining and at the same time enlightening read. I will not come up with yet another summary, you will find plenty of those already. Instead I want to point out a few interesting links to other areas.
Let me start with an interesting connection to my article about personal objectives. One of the statements I had made there, was that a global optimum cannot be reached by local optima everywhere; some local deficiencies would need to be accepted to reach the overall goal. (Apart from the common-sense aspect, this is also formally proven in systems theory.) Well, exactly the same point is mentioned in “The Goal” when Dr. Goldratt writes that at the end of the day, only the amount of shipped goods, i.e. goods sold, counts. Any intermediate over-achievements (“We beat the robot”), is really completely irrelevant.
Another interesting link is one with the book “Turn the Ship Around” by David Marquet (see this for a quick summary). Mr Marquet makes the point that in his experience a deciding factor for an organization’s (in his case the crew on a nuclear attack submarine) performance is, whether its members work to avoid individual mistakes or to achieve a common goal. Similar, in “The Goal” there is a paragraph about the success of various changes in a manufacturing context. It is basically about the different intent (and Mr Marquet calls his style intent-based leadership, by the way). Instead of trying to reduce costs, the manufacturing team looked at things from a revenue generation point of view.
In the addendum “Standing on the Shoulders of Giants” a short introduction is given into the Toyota Production System (TPS). One interesting comment in this section is, that other car manufacturers which have also implemented a comparable system, have not achieved the same level of improvement. This is attributed to the guiding principle: Toyota focused on throughput (and by that ultimately on revenue), whereas all the others looked at cost reduction. The analogy to that is a study (source unknown as of this writing) about customer satisfaction vs. profit optimization. Allegedly, those companies that focus on customers not only win on this front. But in the mid- to long-term they also constantly outperform those that focus on financials.
A very entertaining, yet serious talk about what kinds of “projects” there are and why you need different people for them.