People sometimes ask me what the number one thing I wished did not happen in the tech development industry. It may be different some days, but often it is:
Due yesterday, needed it yesterday & variations
I have nothing against working hard or working fast. In fact, I have no doubt that is one of the primary drivers working for the software or tech industry. The capability to achieve same or similar amount of work in a shorter time is competition won. The manufacturing industry has showcased how this is true, for several decades. The Lean Process and many offshoots also guide us in how to try to replicate such success in the newer industry of virtual products & services.
Interestingly, the very application of such need-driver of “faster” and “deliver ASAP” seemingly can become a toxic element in the companies that try to harness the Agile and Lean processes – manifested as technical debt in common techie view. The prototype is built quickly, but the final delivered product takes much more and the quality suffers AND the maintenance and support effort needed becomes unexpectedly large – which leads to terminating the product support. Sometimes it can get as bad as leading to an organization’s demise.
Why and how does ASAP-culture become problematic in the software industry? The cause can be summarized into three points:
First, software is complex. Apologies to other industries that may object, but software engineering has become a most complex information industry field. Someone shouts ‘just start coding already’ and the prototype is started – and the architecture becomes a blurred mess of lines and the actual implementation is delayed as a result. One alternative of taking the time to create necessary documents and diagrams would be ideal, but if it’s followed true to its literal dogma, too many projects will not finish within 10 years. Optimal balance between the two is the most difficult to achieve.
Second, software engineering, at its epitome, is near art-form. Architecture in software is so much like creating an art-piece, that hurrying it along or pulling an over-nighter does not guarantee faster arrival or a better result. Not being given enough resources to deliver a near-optimal architecture and embarking on implementation often entails 100 or even 1000 times the resources to make modifications after a certain iterations – maybe even releases.
Third, software is like a living thing. There are several concepts brought in from building architectures and many are very similar. But the difference is that a building’s foundation or structural pillars does not change much for a century, while software architecture becomes more emergent after its release into the wild, being improved and re-worked as needs arise. General ball-park estimate is commonly at 4 times the amount of resources are needed to maintain a piece of software system than the amount to develop, after its release. But if the architecture is not flexible or modular enough, it may end up costing upwards of 10 times that (i.e. 40 times the amount taken to implement the original released version) – and still have difficulty keeping up maintenance or upgrades.
Then is it better to be thorough and develop slowly? Never.
Most global, commercially-successful softwares are not slow, monolithic solutions that have really long update cycles. The foremost utility of software development is to deliver market-ready software faster – or as is the curse and driver of need availability ASAP. Any factor that slows that goal, regardless of methodology, regulations, limitations or process can and should be scrutinized for validity.
In other words, the ASAP need or impact-driven culture to delivery of software is what actually slows down the implementation and eventual delivery of software, and the culture is what also induces other technical debt not apparent at release-time. Some technical debts that are by-products of such culture are:
- Lack or absence of project knowledge, insights and data sharing and collaborative strive to be at a state of tranparency to be at the same page, regarding such.
- Difficult-to-scale, incomplete architecture(s)
- Dependent on very few, key developer resource
- Numerous copies of duplicated code
- Lack of work-life balance and inability to increase capability individually and as a resource-pool
- Ever-increasing cost of maintenance
This sort of result can occur, regardless of development methodology or tools employed. SRS (software requirements specification) format of technical spec or using TDD to front-load automated tests as technical specification of features are both ways to develop software faster, by clearing away what is not a well-defined set of features and definitions of the ultimate desired need from the product. Documentation, code-review, process of development, etc. are all ways to help reduce learning curve on knowledge sharing – so doing away with those steps in hopes of getting the product delivery earlier is essentially impractical (and borderline mis-understanding of how to increase effectiveness of available development resources).
Of course, in order for such tactics to be effective in delivering software fast, a well-seasoned architect is needed, but the software development culture supported by the company, not just the project team, needs to be at a good, acceptable level of maturity.
Very few, if any, people would argue that the bricks must be stacked prior to actually designing the building. Foundation work and design of the architecture must first be agreed upon, to build a building as quickly as possible, without a heap of modifications and other related troubles, later in the building process. By the same token, software needs a some firmness with flexibility and modularity afforded in the architecture agreed upon by stakeholders, to respond to changes in the desired features and build-outs – not to mention facilitating collaborative effort expenditures to ultimately reduce both time and cost of development.
In fact, most developers would agree that limits on time creates many serious issues for arriving at such architecture(s). Even so, there is a distinct culprit factor for accelerating the onset of the culture to work in terms of “ASAP”: it’s the business management and the clients/end-users.
To most project management, short-term result is de-factor standard for survival. Owner-stakeholder of a project has a bit longer window on producing results, but many management have a 2 to 3-year contracts (if not less). Extension or conversion of contract into different terms can become horridly difficult if visible results are not shown in those times. Essentially, management that focuses on 6-month to a year’s results prioritizes marketing and top-line sales – it falls on others to think about architecture or maintenance costs.
If a company has a firm stakeholder role of a CTO, the CEO might not be able to demand that architectural flexibility or scalability be sacrificed for short-term impacts – but most situations do not have this safety-net for due-diligence.
The client attitude can also be problematic. Most end-users demand a faster turn-around on their feature-demands from a regional or in-house development teams, while expectations of a fix on a global-scale product usually hovers at around 6-months, and thankfully, at that. They know from previous experience that the global enterprise solution provider that they’ve worked with before does not release a fix any sooner than that.
The clients sometimes actually demand that the developer resources work on-location at the client premises. Instead of properly organizing specifications and getting used to developing software collaboratively via modern communication channels, clients often want to have the developers available on-demand and develop as need arises, to prototype as-fast-as-possible.
The business is used to the culture of impact-drive, client-flavored solution being the highest priority in external parties providing software solutions that are customized. Such experience is often times a barrier to scaling the software solution. The business may have achieved number-1 rating in limited regional scope with such approach, but they often face significant challenges scaling the same solution globally, or to global coverage needed for a client.
Having seen such smaller-scale solutions fail, when delivered as a result of an ineffective, ineffective culture-environment in the solution provider, one must wonder how different the expectation of an issue turnaround might be, by a client used to dealing with robust, global enterprise solutions. Not all, but many would be surprised when there is a response that indicated that a bug will be fixed in a day or two. This would be especially true, if for a critical system – possibly leading to a decrease in confidence-level about the solution provider.
Many smaller-scale development software companies fight through lack of clarity to promise a better fitting solution to the client – and the end-result is an overall shortage of capacity in real terms. The actual software developers end up having to pay for such lack of resources. Developers must quit being able to balance work-life times and end up burning the mid-night oil over longer periods of time, prior to the promised delivery date. There is no doubt the developer talent has no room to grow, in such a situation.
Culture change being difficult is simple-hard fact. It cannot be successful unless the client, the management and the developers all ascribe to the same change in the culture – and foster near-sameness in expectations in the process, as well as deliverable and timelines. It’s simply impractical to expect to change the world, as a single developer. But the first step is by communicating the need to change. Change must also be facilitated by starting with realization about the need to change. Someone must make the sacrifice of being the first one to change and convincing others to follow suit.
But if one developer is the only one that employs the vision of the changed culture and other associates and stakeholders do not embrace the culture change, the expectation would not have changed. Therefore, it’s clearly the highest priority for the business to realize how important architectural engineering is – and the first step can be to enhance the capability of the architectural stakeholders for the projects. The business must do its utmost to balance first-to-market impact-driver and the development/maintenance driver – so instead of just fast, properly-but-fast must be the solution perspective.
Foundational tools and frameworks, methodology and process need to be balanced. It also helps to look at maturing various other development lifecycle cultures and work-flows as a way to fix the effects of being short-termed fulfillment culture. There is no way except to start at least at a single point in the project deliverable.
It can also become a question about chicken-or-egg-first – but just recognizing that development culture maturity is actually an important, high-priority issue is in itself the right first step. Culture is not documentation or mandate – but something that must be lived-through experiences and learning from those with prior experience in changing the culture can significantly reduce the time vested in trial-and-error. It’s a way to improve continually, not just arriving at a path to stop progressing.
Waterfall or Agile or otherwise, that’s why it’s a method/ology, as opposed to a solid piece of scripts that are followed in developing software, for the modern world.
offthemesh 