While watching the video below, I was intrigued, when this interview questions was discussed (timestamp 56:30-57:35).
It made me think about my response, had I ever been asked the same. And it did not take too long before the answer was clear: the B-Tree. There is a very good section on it in Martin Kleppmann‘s book “Designing Data-Intensive Applications“. And I highly recommend this book anyway.
But as a starting point on B-Trees the following video is also quite helpful:
I just came across another interesting statement about the size of development teams:
What makes this so interesting is its origin. It was said in 1968 during the famous NATO Conference on Software Engineering. Fifty years later I think this is still remarkably true. But why?
In my post on the size of development teams I was aiming at complexity and scope as drivers. The interesting question, though, is whether there are additional points to consider. Having worked pretty closely with some other projects in the meantime, I came across another influencing factor: availability of qualified staff.
You can look at this in several ways. The first hurdle is obviously to have enough developers with the required experience at your disposal. The second is how to allocate their capacity to the portfolio of activities you run. Or put differently: How do you deal with the shortage of know-how you face? Because, let’s be honest, there is always more demand for highly skilled developers than can be supplied. (I am having a déjà vu with my “beloved” macro economics lecture 😉 back in 1996.)
If your organization is above a certain size, chances are that overall a sufficient numbers of good people work there. Does that put you in a better position than some poor fellow working for a small company? Not really. Because your project competes for these folks with all the other work within the organization. And what is the difference compared to competition with an external market? Yes, if you have really good connections to higher management, you can escalate things and possibly get additional people. But you will also burn bridges in the process, which is usually too high a price.
So instead you probably end up with the usual mix of folks: Very few rock stars, many middle-of-the-road folks, some promising newbies, and the occasional looser that nobody wants. Can you deliver something really great with such a mix? It will be difficult in a typical corporate setup where you have to somehow involve everybody. And this involvement of the less qualified half of people will slow down the “upper” half.
It is a very delicate subject and there are many fine lines, some from a legal perspective and many more from decency-to-others point of view. Also, the organization needs to think about tomorrow and therefore must have a “funnel” of to-be-rockstars, which need the best training they can get. And the latter is always working on a difficult project with experts and learn from them. But what few organizations do, is look at competence levels in detail and factor them in.
In other words: You can learn a lot from someone who is one, two, or perhaps three levels above you. But if someone is ten years ahead of what you currently know, the difference is just too big. You will only grasp a small fraction of what they teach you, and even that with a considerable risk of misunderstandings. And they honestly cannot understand why you are not following their great advice. Mutual frustration and dislike are usually the result.
Whether you take competence levels into account or not, considerable effort needs to be spent on non-development activities. If you have a taste for management and leading people that will be a great opportunity for you. But if your primary concern is getting something great and possibly visionary delivered, you should seriously consider a totally different approach: an underground project.
Flying under the radar can really be liberating. This must not be confused with idling around or working on some obscure pet project. It is truly about delivering what the organization needs but cannot accommodate in its own structure. Of course you should be certain that your boss will not fire you, if he or she finds out. But as you probably already guessed, driving an underground project does not mean that you are freed from politics, lobbying etc. On the contrary! You must prepare upfront quite carefully how you counter resistance or outright attacks. And yes, you are running a personal risk. But there is no reward without risk.
I cannot say how often I read in some company’s image brochure that management and expert career paths are treated equal. There must be companies on our planet where this is true, but I have yet to see it in practice. Why is that? (Unless mentioned otherwise this article is about the IT industry with a strong focus on consulting and software companies.)
In my experience the core reason is a fundamental lack of understanding from the non-technical folks that make the relevant decisions, i.e. management. They have a vague impression of techies that is often dominated by perceived social deficiencies (“They can’t look me in the eye”). While the latter is sometimes true, how can it be a measure of someone’s qualification to perform a complex technical task? And perhaps there is actually a good reason for averting my eyes. I cannot speak for others, but when I have a complicated discussion, it helps me focus all my mental capacity on the problem at hand when I look onto the table.
I would speculate that, as in so may other cases, the heart of the problem lies with mid-level management (where the details are determined) as opposed to the C-level. Taking into account the Peter principle, what does that say about middle management? Yes, this a strong simplification and it does many people injustice. I have personally met quite a few people on this level that I consider great at their job, particularly including leadership skills. But just as well did I encounter folks who cannot think out-of-the-box and blindly follow rules and processes. And it is this capability to see new ways, make realistic projections about what will (not) work, and lead by example “where no-one has gone before”.
A particularly interesting story for me was the encounter with an HR person where we talked about setting up an initiative to advance technical folks (what is often called a high-potential program). My idea was met with friendly resistance and the argument was made that contrary to management, the technical side is so much more diverse that it would be virtually impossible to have a single program. In fact this HR person seriously thought that every techie needed his or her completely individual program.
My counter argument then was that this would not be a technical program (learn programming language XYZ) but one for professional and personal development. So we talk about e.g. how to present a complex technical scenario to upper management for getting a budget approval. Or how to engage with a customer and instill confidence that a given product is the right choice to solve their business problem. And all this with being authentic and building upon the strong technical knowledge one has.
I don’t want to appear as simply bashing HR here. But the underlying problem was that assumptions had been made about what would be suitable for technical folks without ever having spoken with any of them. Would this happen the same way for management staff?
From a more abstract perspective it comes down to the fact that many decisions are made based on personal trust and relationships, rather than on processes and policies. So what is needed is that the relevant management and technical staff (on every hierarchical level) talk to each other in the spirit of equality. And here both sides fail miserably too often.
I have recently watched a very interesting presentation by Gunter Dueck (former CTO of IBM in Germany), where he talks about the future of work. When looking at how many consulting organizations structure their business, you can see a trend that seemingly high-level parts, usually the interaction with the customer and where business domain knowledge is required, stay in your own country. And those aspects that can, allegedly, be performed by less qualified people are moved to countries with lower salaries.
This may seem attractive from a cost-management point of view for a while. And it certainly works right now – to the extent possible. Because the organizations still have people locally who had been “allowed” to start with relatively simple coding tasks and grow into the more complex areas of the business. But what if those folks retire or leave? How do you get people to the level of qualification that allows them to understand the customer’s business on the one hand, and at the same time be able to have an intelligent conversation with the offshore folks who do the coding? Because for the latter you must have done coding yourself extensively.
There is a widespread notion, that developers at some point in their career evolve into something “better”, called architect. This term has the connotation of mastery on the subject, which I think is ok. What is not ok for me, is that in many cases there is the expectation that an architect’s time is too valuable for something as mundane as coding. Instead architects should develop the overall architecture of the solution.
Ok, so what is the architecture? Most people believe that some documentation (often a mixture of prose and high-level pictures) is the architecture. I would argue that this not the architecture but a more or less accurate abstraction from it. When enough work has gone into things, it may may even become the to-be architecture (but certainly not the as-is one). However, outside of regulated industries I have never seen a document that was thorough enough. A friend of mine used to work on embedded systems for car breaks where lives are at stake; and he told me some interesting stories about the testing and documentation efforts these guys take.
In my view the architecture is, by definition, in the code itself. Everything else is, I repeat myself, something that has some relationship with it. So how can an architect not be coding? You could argue that instead of doing the work him- or herself, mentoring and guiding less experienced developers is a good use of the architect’s time. For me that works fine up to a certain level of complexity. But if we talk about an STP (straight-through processing) solution, is that really something you want to give to a mid-level developer? (It would probably be an ideal piece of work for pair-programming, though.)
I do certainly not want to demean people who call themselves architects. It is a not-so-common capability to look at an IT landscape and see the big picture. Instead many people will be lost in the details. So we definitely need this this perspective! But it is a different kind of architecture, the so-called Enterprise Architecture (EA). I know folks who started as (really good) developers and are now very successful at Enterprise Architecture.
So, in closing, my core point is that the architecture of a solution and its code are basically two sides of the same coin. Everybody involved on the technical level should understand both aspects. And if the level of detail varies, depending on the official role, that is probably ok.