Somehow there’s this idea that Architecture and Agile are conflicting. In my opinion, architecture is an enabler for agility, not an obstacle.
Agile and architecture deal with two different aspects of software development. Agile deals mostly with project management, architecture deals with solution constraints.
Agile is basically a solution for the problems caused by waterfall project management. It solves these problems using short iterations, fast prototyping, early testing, fast failure, allow for change of requirements based on feedback, planning based on measured productivity (earned value).
Agile is often mixed up with Lean, but Lean has a slightly different vision: making your value stream as efficient as possible. Ways to achieve this are waste elimination, deciding as late as possible, minimising inventory, include learning, etc.
Architecture deals with the constraints that your solution has to fit in. This doesn’t mean all technical design decisions have to be made upfront. There are different types of architecture. Not all of these need to be completed up front. Architecture does not mean Big Design Up Front.
Most architecture you need when creating software is different from building architecture. It’s more like urban planning. Urban planning includes zoning, but also building regulations, safety requirements and environmental requirements. These are all constraints which define the solution space for the building architect. Architects can still create the most amazing and original buildings within these constraints.
You probably need some software urban planning before you can start your agile software projects, depending on the situation your solution has to fit in. You probably need to decide on your components (zoning). You might also want to choose a platform, and a way to communicate between components. And maybe a way to store data. And some other constraints. But within these constraints you can still do Just in Time technical design (solution architecture) whenever you start working on a User Story.
Lean specifies that you need to make irreversible decisions at the last responsible moment, but not later. Is it responsible to create every user story on a different platform? Is it responsible for every component to use a different communication protocol? It’s a waste to re-evaluate your platform for every user story.
When you get these kinds of architecture decisions out of the way, you can focus on delivering business value by implementing user stories. The right set of constraints will enable you to deliver user stories with a higher velocity.
Having the right set of architecture constraints will also allow you to postpone the technical design of user stories until you actually start implementing them. A bit of component/information/enterprise architecture enables you to do Just in Time solution architecture.
So the problem isn’t architecture, but the waste that current architecture procedures introduce into your value stream. Try to optimize your value stream without throwing out architecture completely.