PMO Focus | Navigating the Transition to Agile: Gorillas in the Backlog by Allen Fordham
An important potential obstacle to digital modernization is the change in team social dynamics that accompanies a transition away from waterfall towards Agile software development.
As teams maintain and modernize digital resources, it is very likely that they will at some point implement a form of ‘Agile hybrid’ approach. This is a logical and valuable step toward identifying a modern project management solution that fits the team’s specific needs. But transitioning to Agile Scrum project management can put pressure on team members. This shift in methodology requires not just learning new tools and adopting a very different day-to-day rhythm, but also means that every team member needs to redefine their relationships and interactions with their colleagues. This transition can be especially difficult because of the social expectations that accompany a team’s Agile transition.
Silverbacks in the Waterfall
Digital teams using Waterfall project management have often been using the same tools and workflows for decades, and the status quo has persisted because it has worked. These teams tend to be hierarchical, with each team having a Team Lead who identifies requirements, allocates work, and supervises development. The most experienced or capable member of the team is responsible for moving the project forward. Decisions are routed through this leader, and internal team disputes have a clear arbiter.
If the development team can be compared to a troop of wild gorillas (and this can often feel apt), this distinguished, grizzled, capable team lead is the team’s ‘silverback’ – the hub of the cohesive group. His or her efforts have contributed substantially to the success of the organization. Everyone on the team knows that the silverback is the one responsible for arbitrating disputes, guiding the team, and leveraging his or her considerable skills and experience to make decisions. If a failure occurs, the silverback bears responsibility. This is an intuitive social structure for small groups of both gorillas and developers.
But the modern digital landscape is growing more complex, and solutions that have been in place for a long time are often poorly-suited to today’s challenges. Every year, advances in security, reporting, regulatory compliance, external integrations, machine learning, hardware, cross-device compatibility, and a host of other specialized technical considerations mean that it is growing harder and harder for traditional teams to clearly forecast the full requirements of a project from the outset. Silverbacks are increasingly faced with unfamiliar challenges, and they must rely on the advice of other team members or partners to remain competitive and keep the organization running smoothly. Development teams are turning to Agile project management to democratize their digital practice, because it helps to ensure every contributor’s value is realized and allows projects to adapt to suit complex changing needs.
Agile teams, in contrast to waterfall teams, are explicitly anti-hierarchy. There is no team lead on an Agile team. Instead, there is a Scrum Master serving as a ‘servant leader’ who must focus on facilitating and empowering every contributor instead of delegating, assigning, and approving work. But the Scrum Master is not a boss. The entire team shares responsibility for the product’s quality, not just the Scrum Master. Scrum ceremonies are designed to encourage collaboration, openness, and trust to maintain high-quality output. Disagreements are resolved by discussion and consensus. There is no need for a single arbiter on an Agile team; in fact, maintaining an authoritative presence can be counterproductive, as the team needs to rely on a broad range of team member specializations to meet the organization’s digital needs. As individuals, we can adeptly navigate consensus-based team social structures as well. But modern businesses, for reasons that should be clear, have seldom fully embraced this philosophy, which is one of the reasons an Agile transition can be challenging.
The Problem of Social Inertia
As organizations seek to modernize their digital practice and begin to transition their teams away from waterfall and towards Agile, the existing dynamic between authoritative team lead and subordinate developers must be overcome. However, this is an unnatural process for us. Our intuitive ability to recognize the power structure within a small group and adjust our expectations and behavior accordingly has helped shape society and civilization since before recorded history. Humans, as individuals, are of course capable of comfortably operating across the spectrum from highly egalitarian circles of friends to highly hierarchical military units. But we don’t tend to be good at mentally reframing existing group dynamics overnight. This resistance to changes in stable social or group dynamics is known as social inertia.
Social inertia refers to the tendency of groups (like project teams) to resist social change and preserve the status quo. This is useful - for both gorillas and humans - in maintaining social order and stable relationships. But social inertia presents a challenge for organizations seeking to modernize their digital practice, because the organization is deliberately redefining the team’s firmly-established hierarchy to better respond to modern conditions.
Social inertia is difficult to quantify. Agile teams commonly track metrics like story points completed per sprint to roughly measure a team’s velocity and performance; there is not yet an objective metric to track how quickly small-group social hierarchies are restructured. The extent that social inertia will affect each team is also difficult to anticipate, because all teams are made up of individuals with different collaborative tendencies.
How Does it Happen? (Or: “The Team is Going Bananas!”)
It makes a lot of sense to keep existing teams intact as the organization begins to adopt agile principles. After all, the current team is probably already familiar with the software. Interpersonal conflict is likely rare (or at worst, manageable), since everyone understands the hierarchy, and the resident silverback keeps things in check. Besides, transitioning development workflows and establishing the cadence of daily work is complicated enough without the additional mental and emotional overhead of getting to know a new team and catching up on unfamiliar legacy software. So when organizations begin transitioning from waterfall to Agile, existing teams are often left in place but ‘rebranded’. The organization pays for team training, orders a name plaque for the Team Lead’s desk with the new title of ‘Scrum Master’, and waits for the promised productivity increase to manifest.
And just like that, the transition to Agile is in trouble. Keeping the team intact but dramatically changing the social expectations of the group leader has introduced a significant disruption to the team’s social inertia. This experienced manager is accustomed to leadership and delegation, but is now expected to serve his team as a peer. The other team members of the team are suddenly expected to regard their former esteemed silverback as an equal, a ‘servant leader’ with limited authority to command.
This is a confusing situation for the team members. Even in the best of circumstances, when all team members stay committed to the organization’s vision and the new structure, it will likely take months for the existing interpersonal power relationships to meaningfully change. In the meantime, leadership can become frustrated that their Agile team isn’t performing as expected. The speed and quality of a team’s output decreases, and morale takes a hit, sometimes resulting in valued contributors looking for other opportunities. Agile methodology itself is often blamed, when the problem lies in the social ramifications of the implementation.
The 800-Pounder in the Room (or: “Stifled Retrospectives”)
To see why Agile can be stymied when old hierarchical power structures remain intact, look no further than the sprint retrospective.
The retrospective, or ‘retro’, is one of five standard scrum ceremonies. Teams using Agile Scrum will typically hold a retro every couple of weeks. The goals of a retro are to identify strengths and weaknesses of the tools, team, processes that emerged during the sprint, and to discuss improvements to be implemented and further refined in the future. As the team starts down the road of Agile implementation, healthy retros are a crucial part of continually refining the details of the team’s tools and empowering the team to collectively fine-tune the most productive implementation of Agile to match their unique skills, resources, and goals. A team with productive retros will be more ‘agile’ in a sense that it allows the team to identify impediments and adapt quickly in response to evolving requirements and capabilities.
But there’s a catch. Retros only add value if the team can have open, honest discussion about their strengths and struggles. If improvements aren’t collectively identified and embraced on an ongoing basis, then your team isn’t agile at all; it’s just a traditional team with a few extra meetings. Not even the highly capable silverback can hope to identify, triage, and solve every pain point faced by every team member alone. The team must work together.
In my experience, open, honest retrospective discussions come naturally for egalitarian Scrum teams. Sitting at a table with peers feels like a safe space to commiserate about challenges that the team is facing together, and everyone feels motivated to participate and contribute. But once ‘the boss’ comes in and sits down, the room gets quieter. Team members are disinclined to openly challenge a decision or process that was implemented by the supervisor sitting next to them. Even though everyone has been told that the previous boss is now a peer, it’s a genuine challenge to internalize the new social model when everyone is still sitting in the same places around the same table as before.
The consequence is that retrospectives are awkward, muted affairs. Their full value isn’t realized, and they begin to feel like a waste of time. Without ongoing collective retrospection, it is very difficult to collect honest, actionable feedback on how the process and project are doing. This feedback is crucial to continually refining the workflows to meet the unique needs of your team. This means that the team’s Scrum implementation is less adaptive, and impediments to productivity linger longer than they should.
Reducing the Impact to the Team (or: “Ape it ‘til you Make it”)
Retrospectives are only one example of how social inertia can stifle an Agile team’s productivity. Agile performs best when every team member has a feeling of ownership in the process and delivery. Transparency is key; the goal of the project is clear to everyone from day one. Developers work together to refine the backlog of work to be done, collaborate to determine workflows that meet everybody’s needs, and individually volunteer to pick up tickets that help the project progress toward completion. Completed work is reviewed by peers and then merged into the codebase as the project matures. Success is achieved as a team, not individually. Scrum masters must encourage and enable this mindset, but this can be difficult when existing hierarchical social structures are not thoughtfully restructured.
To help reduce the negative effects of social inertia, leadership should consider a temporary or partial shuffle in the makeup of existing teams. With every change to team makeup, the existing social inertia is disrupted. Think of this as a gentle ‘reset’ button on social interactions. This must be done thoughtfully, but can be less disruptive than overcoming deeply-ingrained social inertia. A shakeup will help the team embrace an Agile-friendly social structure more readily. This will help to ensure everyone’s feedback is collected, heard and acted on. When team members realize that their voice matters, they can feel more of a sense of ownership of the process and the project. Quality of output increases as trust is built and the team refines its tools and workflows together.
If you can, bring an experienced scrum master to guide the team through the transition from waterfall to agile. A new personality in the room helps to spur everyone’s readiness to redefine their own position on the team, with the added benefits that a seasoned scrum master is accustomed to the role of servant leader and is also likely to have learned from the mistakes of other teams in a similar situation. Consultancies like EQengineered can provide bespoke support to teams modernizing their digital practice. When selecting a partner who’s a good fit for augmenting your team during its Agile transition, seek someone who has experience modeling good Agile habits, can coach teams to own the process themselves, and who cares about the dynamic of the team.
Substantial team restructuring or hiring new Scrum Masters isn’t always prudent or viable. If shuffling teams doesn’t make sense, a team may find it beneficial to ask the silverback to not attend sprint retrospectives, at least initially. Empower a capable peer to guide retrospective discussions as the team gets comfortable with the principles of Agile. One drawback of this solution is that the team misses out on the perspective of the most experienced team member while holding the retro discussion. This also doesn’t address the difficulty in encouraging individual ownership of deliverables and processes.
Regardless of the extent of team restructuring, it is vital for everyone on the team to have an open discussion about how roles and responsibilities in Agile differ from previous norms. Talk through the principles of Agile project management, and emphasize transparency, openness, and respect.
Conclusion | Take Aways
Transitioning from a traditional Waterfall approach to an Agile methodology requires not only adopting new tools and processes, but also redefining team dynamics and relationships. Understanding and addressing social inertia is vital for a successful transition, as it directly impacts team collaboration and productivity. By recognizing the potential obstacles and planning for them, organizations can better navigate this transformation.
Leaders must be willing to disrupt existing hierarchical structures, promote open communication, and create an environment where every team member feels valued and heard. This might include reshuffling teams, bringing in experienced Scrum Masters, and ensuring that retrospectives are conducted in a way that fosters honest feedback and continuous improvement. Whichever solution or solutions you decide to try, it is crucial to clearly define and openly discuss expectations with every individual team member and stakeholder early in the team’s transition to Agile.
Most importantly, be kind to those who have played a role in your organization’s success and provide support to them as you shape your team’s Agile mindset together. Their experience and understanding of legacy systems will remain vital to your organization, and their value will be magnified as they collaborate, share their knowledge, and contribute to projects alongside their peers. Make sure to check in with them regularly to ensure that they know it.
It is impossible to predict the ‘final form’ that any newly-agile Agile team will take to meet the unique challenges and capabilities that they face, but by applying these tips and keeping the principle of social inertia in mind, you can give your team a head start on their journey. Ultimately, the goal of transitioning to Agile is to create a more responsive, adaptable, and collaborative team environment. By thoughtfully addressing the challenges of social inertia and fostering a culture of transparency and trust, organizations can harness the full potential of Agile methodologies. This not only enhances the quality and speed of project delivery but also boosts team morale and engagement. Embracing these changes can lead to more innovative solutions and sustained success in the ever-evolving digital landscape.