I had wondered about what having an ’emergent’ architecture experience would be like.
To get the full experience, it took two different products and about three years of real time.
First year was about getting the proof-of-concept out the door, focused on the look-and-feel of the front-end component.
The POC was really a reincarnation of a failed project, the result of not adapting the original waterfall design to the changed technology landscape (cost-effective big-data products, for example) well, mostly ending-up in numerous pivot attempts by PM. PMO was unable to recognize the gift of gab. So now we moved to a new roadmap.
Having chosen a big-data database component, we moved to use the flashiest API tech, node.js. Here we hit a roadblock. Although the tech team wanted to validate the vision of continuous integration and delivery, we fell short on being able to execute BDD/TDD approach.
Dropping the test-driven requirement from the project, we moved to settle on an initial architecture that put data-flow and presentation on separate threads of work. Of course we would’ve preferred separate teams, but reality was that we really only had enough time to bring in couple front-end experts. The features were built by employing an approach that resembled Scrum, but modified to fit our company’s state and CMMI maturity level. (scrum-but?)
I decided to accept parts of the applications that seemed to violate my want of an API-centric architecture, because I wanted the POC to come to fruition more than having structure I like.
I struggled with it on a daily basis, but letting it go was better for our environment than enforcing some rules.
The end-result had the look, and the capability to handle most of the workload without too much issues. There were a myriad of issues, both functional and not. We knew going in that the application wouldn’t get much use this first year. We released it to two clients. It wasn’t much of a system, just an app backed by a prototype API. Even the data flow was not fully implemented, able to import a big portion of the datasets, but somewhat hacky and not exactly reproducible at the frequency we wanted. This is how the first year came to pass.
Second year was half spent in supporting the release, short-handed, as the team took some loss in resources – due to the death march prior to the release. But after a few initial months, there was opportunity to work on a new product, using the same system. Many months afterwards were spent in PMO planning.
Luckily, the previous release did not get much use, by the clients - taking less effort to maintain.
The spec came as a almost-a-CRM system that uses a rework of the previous product’s front-end component, inside a web-app. The spec also called for a near real-time customer feedback aggregation and charting -the ‘big data’ aspect of it.
Third year the team built the next version of the system, which uses the previous product as a basis to improve upon.
This it’s where it got interesting. With an actual due date for delivery already set, we had to work with a new data source as the primary data and the CRM aspect added on workload we had not worked on before. We were lucky to have a senior dev who had experience with a public-facing CRM – the work involved and how it may fail grotesquely, if not carefully carved-out.
After a month, it became clear we wouldn’t get the complete spec in time. The management called for vertical implementation of the features, in answer. However, this meant we had to adapt our architecture, as new features became clear enough to work on. So here is where the real meat of this post is.
Architecture that came with minimal pre-planning, answering minor challenges with grace, but doing our best to make sure no sweeping change will be needed. Going in without some core systems design would be opening ourselves to painful nights of haphazard changes and unnecessary risks.
