Scaling Agile Development Via Architecture

Every system has an architecture, even systems developed using agile methodologies. Whether you attempt to define that architecture up front in detail or whether it emerges over time is up to you. My experience is that most agile teams follow a strategy somewhere between these two extremes, one that involves investing a bit of time up front to think through the "big issues" but which addresses the details on a just-in-time (JIT) basis. That strategy, combined with proving your architectural ideas as soon as possible through working code, results in a very agile and low-risk approach to system architecture. This article summarizes a collection of strategies for addressing architectural concerns on agile projects and discusses how such strategies can be applied to scale agile methods to large development efforts.

Agile Architecture Strategies

Many of the architectural strategies listed below describe trade-offs; the implication is that you must tailor an approach which meets your unique situation. These strategies are:

1. Model with others. Modeling, architectural or otherwise, is best performed by two or more people working together collaboratively. By working collaboratively you will create a higher quality product, will develop a shared vision, and will learn from one another.
   


2. Focus on collaboration over documentation. {sidebar id=1} "Agile architects" are active members of development teams who also write software -- they are not simply people who document their vision and hand it off to developers. This greatly increases the chance that the system reflects the desired architecture and that the architectural vision reflects the realities faced by the team. Note that there may still be some documentation, but that's a secondary effort.
   


3. Prove it with code. Everything looks good on a whiteboard, or in a PowerPoint slide deck, or in a modeling tool. You have no idea if your architecture actually works until you try to implement and test it. The implication is that you should prove that your architecture works, something that Extreme Programming (XP) calls spiking and the Rational Unified Process (RUP) calls architectural prototyping. Sometimes you will discover that your original approach doesn't work or how your approach actually works (instead of how you thought it would work). Proving your architecture with code helps to reduce risk to your project because you quickly discover whether your approach is feasible; the longer you go without coding the greater at risk you are.
4. Keep it simple. Agile software developers model, but we often do it in ways which are very different than traditionalists. We create sketches, we capture requirements in index cards, we create abstract user interface models using paper and sticky notes, we write acceptance tests which we consider to be requirements, and we write unit tests for our detailed design efforts.
5. Use the simplest tools. The vast majority of agile developers create free form diagrams, often simple sketches on a whiteboard which overview how we intend to build the system. These diagrams evolve over time. Teams lucky enough to have dedicated whiteboard space will evolve their sketches appropriately as the project progresses. Sometimes the diagrams are captured and cleaned up using a drawing tool or a software-based modeling tool, and we may even generate new code from the models or use the tools to visualize existing code. Sometimes the simplest tool for the situation at hand is a whiteboard; sometimes you need a tool that is a bit more complex.
6. Think through the big issues up front. Figure 1 depicts the Agile Model Driven Development (AMDD) lifecycle. It indicates that initial architectural modeling is typically performed during "iteration 0," the initial inception phase of your project. For short projects (perhaps several weeks in length) you may do this work in the first few hours and for longer projects (perhaps on the order of twelve or more months) you may decide to invest a few days to do the initial architecture modeling.
7. Think through the details just in time. As you can see in Figure 1, you will often choose to "model storm" focused issues on a JIT basis. These model storming sessions are typically impromptu events: one project team member will ask another to model with him, typically lasting for five to ten minutes (it's rare to model storm for more than thirty minutes). The people get together, gather around a shared modeling tool (e.g., a whiteboard), explore the issue until they're satisfied that they understand it, then they continue on (often coding). It is desirable to take a JIT approach to identifying the details because the requirements are going to change over time anyway, you can ask better questions based on your greater domain knowledge later in the lifecycle, and your stakeholders provide better answers because they've seen working software delivered on a regular basis. Frankly, there's no rush - if you have the ability to model something today you will have that same ability tomorrow.
8. Allow good architectures to emerge over time. Although you can start with an architectural vision, or an architectural metaphor if you like, the fact is that the details will emerge as your system evolves to meet the changing needs of your stakeholders.
   
9. Travel light. Don't create a 50-page document when a five-page one will do. Don't create a five-page document when a diagram will do. Don't create a diagram when a metaphor will do. Remember Agile Modeling's They Ain't Gonna Read It (TAGRI) advice: close collaboration with others is a much more effective way to communicate your ideas than handing them documentation.
10. Have a few overview diagrams. Just because you're traveling light doesn't mean that there isn't any documentation at all. I typically strive to create one or more navigation diagrams, diagrams that present an overview of the "landscape" of the system. Just like a road map overviews the organization of a town, your navigation diagram(s) overviews the organization of your system.
11. Be flexible. The type of navigation diagram(s) that you create depends on the nature of the system that you are building. No one set of architectural views is right for every project. Instead, the nature of the project will help to define the types of views that you should consider creating.
12. Display models publicly. Your architectural diagrams should be prominently displayed where your team can see them and, better yet, update them. For a co-located team your best approach is to simply have dedicated whiteboard space for the team. Distributed teams find that a Wiki with snapshots of diagrams and point-form text works well.
13. Take a requirements-driven approach. Your architecture must be based on actual requirements put forth by your stakeholders, otherwise you are "hacking in the large."
    
    
    

Figure 1. The AMDD lifecycle.

Architecture Enables Scale
Although many software development projects are done by teams of less than ten people, some are much larger. Worse yet, your team may be spread out amongst multiple locations, time zones, and even organizations. To deal with the inherent complexities of managing such a team you will organize it into sub-teams, which in turn may be recursively organized into more sub-teams as required. These sub-teams will typically be responsible for developing one or more subsystems, and they must ensure that their work integrates easily with the other subsystems. This motivates you to have a core architecture team which identifies, supports, and evolves a common architecture across the various sub-teams.At the beginning of a large project you should identify your most experienced developers and abstract thinkers, as well as a few people that you want to see get some architectural experience, and invite them to be members of the core architecture team. You want to do this for two reasons. First, you want good people. Second, each of the development sub-teams will include at least one member of the core architecture team. This increases the chances that each sub-team understands and follows the overall architecture and that the core architecture team will not ignore portions of the system. Furthermore, it ensures that each sub-team has some senior people on it with an understanding of the architectural vision. Frankly, this is something you would do on any large project, agile or not. Agilists are very practical, we're more than happy to adopt good ideas from the traditional community.The core architecture team is responsible for identifying the initial architecture then bringing it to the rest of the project team for feedback and subsequent evolution. Architects will typically do this during iteration 0 of the project, which for larger projects can prove to be several weeks in length. Their goal is to identify the subsystems -- critical information required to identify not only the potential sub-teams but the skills required on those sub-teams. At this point in time you also want to identify and agree to the interfaces to these subsystems. The interfaces will change over time, so be prepared to refactor your code accordingly, but the goal is to enable the sub-teams to focus on their portion of the system without having to worry about what the other teams are doing. This strategy is something that Grady Booch calls "managing to the seams."[1]To avoid an ivory tower architecture, the members of the core architecture team take active roles on the development sub-teams, communicating the architecture to other members of the sub-teams and working with them to prove portions of the architecture via concrete experiments. From the point of view of the development sub-teams, the architect acts as both an architectural consultant and as an active member of the sub-team. In other words, the architect is another member of the team who gets his hands dirty coding.The core architecture team will find that members need to get together occasionally to evolve the architecture as the project progresses, negotiating changes and updating any architectural model(s) as appropriate. These working sessions will be frequent at the beginning of a project and will be needed less and less as the architecture solidifies. It will be common for members of the development sub-teams, who may not be members of the core architecture team, to be involved because they may have some specialized knowledge or experience to share with the team. The best meetings are short, no more than an hour in length, and are often held in a room with lots of whiteboard space. Everyone should come prepared to the working sessions, willing to present and discuss their issues as well as to work together as a team to quickly come to resolutions.You may even find that you need to scale to an even more difficult situation: some projects are part of a larger program which may comprise several dozen other projects which are implemented over a ten to fifteen year timeframe. Your core architecture team and the models which they create will help to promote a shared, evolving vision throughout the lifetime of such an effort.



Parting Thoughts

You'd be hard pressed to find an agile team that doesn't model on a regular basis, and this includes architectural modeling as well. This article overviewed a collection of architectural strategies for agile software development teams. I invite you to join the Agile Modeling mailing list, www.agilemodeling.com/feedback.htm, so that we can continue to discuss agile approaches to architecture, requirements, and design as a community. I also invite you to get involved with the Eclipse Process Framework (EPF) at www.eclipse.org/epf/ where we are developing both an open source editing tool and open source process materials describing agile development methods and techniques. There is a lot of interesting work going on around agile architecture techniques and I'd love to see you get involved. I'd like to thank Per Kroll and Jim Ruehlin for their feedback regarding this article.



Suggested Resources

The Agile Data Site, http://www.agiledata.org/, contains several articles describing agile approaches to enterprise architecture. The Agile Modeling Site, http://www.agilemodeling.com/, contains a large number of articles about agile modeling techniques, agile requirements, and agile architecture.The International Association of Software Architects (IASA) Site, http://www.iasahome.org/, is a great resource for IT architects. Grady Booch's Architecture blog, www.booch.com/architecture/blog.jsp, is a great resource for anyone interested in leading-edge ideas concerning architecture, Service Oriented Architecture (SOA)s, and software development in general.IBM developerWorks Architecture Forums and Community, http://www.agilejournal.com/www.ibm.com/developerworks/architecture/community/, offers a wide range of resources for web services, SOA, J2EE, UML, and architectural techniques.
[1] See Grady Booch's book Object Solutions: Managing the Object-Oriented Project, Addison Wesley, 1995.
About the AuthorScott W. Ambler is the Practice Leader Agile Development in IBM's Methods Group and is a fellow of the International Association of Software Architects (IASA). He is the founder of the Agile Modeling (AM), Agile Data (AD), Agile Unified Process (AUP), and Enterprise Unified Process (EUP) methodologies. Scott is the (co-)author of several books, including Refactoring Databases, Agile Modeling, Agile Database Techniques, The Object Primer 3rd Edition, and The Enterprise Unified Process. Scott is a Senior Contributing Editor with Dr. Dobb's Journal and his personal home page is www-306.ibm.com/software/rational/bios/ambler.html
Written by Scott W. Ambler Thursday, 09 November 2006 06:43

Four Strategies for Task Allocation

Iteration and release planning are keys to successful agile projects, but overall have a relatively small impact on a developer's day-to-day life, compared to the daily planning that takes place each morning. The strategy a team uses to sign up for work has significant implications for what a developer's day will look like, impacts his work style and habits, and ultimately can significantly impact the overall success of the iteration. Unfortunately, the agile community gives relatively little guidance in this area. In this article, I will share my experiences with three strategies for task allocation, drawn from several typical agile projects with two to three week iterations.

Strategy 1 - Ad Hoc
The simplest approach to selecting tasks is ad hoc. Any developer can sign up for any task he feels like working on that day.This strategy has much to commend it. Since it maximizes flexibility, in theory this should provide the highest potential velocity. The team can sign up for work in whatever order makes the most sense. If a task is blocked for some reason, the ScrumMaster can take responsibility for removing the block, and the rest of the team simply picks up tasks that are not blocked. Higher risk items can be addressed earlier. Developers can choose tasks based on their energy that day, their interests, and any other relevant factors. It is also fairly easy to accommodate events such as unexpected absences by team members.This method also supports a variety of good practices, such as switching pair programming partners frequently (at least daily), least qualified implementer, and collective code ownership. Few silos of knowledge about functionality or architecture develop, and when they do, it's an easy problem to fix.Finally, this approach maximizes the idea of team accountability. The entire team is responsible for getting all the tasks done. We all succeed or fail together. This strategy also has some potential weaknesses, which generally manifest themselves on less mature teams. Chief among them is the problem that a team member can easily slack off, either by "pairing with" (read: "watching") stronger developers, or by consistently picking up trivial tasks and exaggerating the time needed to complete them. While this is usually easy to detect, it's quite hard to objectively demonstrate the poor performance or unambiguously define the desired behavior - which makes it tough to manage. While a better approach to pairing will help detect or prevent this, a team may be better off with a strategy that makes it harder for slackers to hide.A second weakness of this approach is the unfortunate tendency for many stories to be in active development at any given time. As developers move from one "juicy" task to another early in the iteration, it's very tempting to leave the less exciting tasks for later. Ultimately this causes many stories to get done at roughly the same time - about 15 minutes before the code freeze. This can overwhelm QA and the Product Owner, who do not have time to accept the stories or adequately test them. The resulting blame game during the demo and retrospective is anything but productive.The third weakness with this approach is a flip side of its strength, team accountability. No single individual is directly accountable for much of anything other than his daily tasks. When everyone is responsible, it can work out that no one is responsible. This can result in a lot of stories that aren't quite finished, but nobody feels particularly bad about it.

Strategy 2 - Dedicated Developer per Story
Another strategy for signing up for work is the dedicated developer/pair per story. A developer, or a pair of developers (depending on your practices), signs up for a story and is responsible for seeing the story to completion.If the ad-hoc approach is the most agile, this has got to be the least. It is the least flexible and destroys any real sense of team accountability. However, it is not without some strengths and other characteristics that might make it appropriate in certain situations.One major advantage of this approach is that it provides nowhere for "slackers" to hide. Each developer (or pair) is responsible for seeing at least one story through to completion. Whatever status he may report at the daily standup, at the end of the iteration, the story is either done or not. This provides the necessary objective evaluation that might be needed to deal with team members who are less than fully engaged on the project, should it come to that.A second advantage is that it causes each developer to gain a comprehensive understanding of the overall design of the system, since each developer will likely need to modify most or all layers of the application during the course of one or two iterations. Any given story may not include all architectural layers, but over time it's very likely a developer will gain experience in each piece.This approach may also work better with teams who can't, or don't want to, collaborate very much. Each developer can focus almost exclusively on developing his story without too much interaction with the rest of the team. This is not the most desirable scenario, but could be appropriate. In particular, a team that is not co-located on a regular basis, or is separated by several time zones, might be more effective with this approach. It may also suit teams who are new to agile and have been used to working independently.This approach, however, has significant drawbacks. By maximizing individual accountability, it effectively destroys team accountability. Together with the minimized necessity of collaboration, this approach eliminates most of the behaviors that make a team, a team. There is no real common goal or cooperation on a daily basis.The "dedicated developer" strategy will reduce or eliminate architectural silos (e.g., only one or two developers understanding the back-end system) but introduces strong tendencies for silos built around particular features (e.g., only one developer understands how the order notification code works).Because one developer is dedicated to one story, if the story is blocked, the developer is blocked. Similarly, if the developer is unavailable (due to illness, vacation, production emergencies, etc.), then the story will not likely get completed. It's unlikely that another developer will be able to come up to speed on the story in time to complete both it and his original one.Finally, this strategy shares a weakness with the "ad hoc" method, in that many of the stories will tend to be completed right at the end of the iteration. It's also possible that several stories will be "almost done" but not completed. If the team has overcommitted, several developers may find they have bitten off more than they can chew, and subsequently all fail to complete their individual stories. Ideally, the team should have realized this, abandoned a story or two, and gotten the rest completely done. It's better to have, for instance, 75% of the stories completely done, and 25% not even started, than have all of the stories 75% done and none finished. The "dedicated developer" strategy does not accommodate this situation, and worse, removes any real incentive for a developer to sacrifice "his" story to help the rest of the team.
Strategy 3 - Story Swarming
Using a "story swarming" strategy, the team "swarms" each story in turn, seeing each one through to completion before starting the next. In practice, it's frequently not possible for all the developers to work on the same story, so there are generally two or three stories being developed at the same time, each by a subset of the team. This practice has many advantages and avoids most of the problems faced by the two strategies previously discussed.Story swarming promotes team accountability very strongly. Although the entire team may not be directly involved in the development of each story, a large percentage of the team will be. This generally avoids the problem of a developer feeling that he is only accountable for delivering his own stories or tasks. Granted, the subset of the team not directly involved in a given story may not feel a strong sense of accountability, but enough of the team does that the concept is preserved. Additionally, by focusing on completing a story before moving on to a new one, this approach ensures that developers feel completely accountable for delivering the stories they do work on, rather than just completing the tasks they happened to sign up for. From this perspective, team accountability may be even stronger using this strategy than with the "ad hoc" approach.Since several developers will work on each story, most of the benefits of "ad hoc" approach are retained. There are enough developers involved with each story to support practices such as pair programming, as well as avoiding architectural silos or silos around particular features. It requires a high level of collaboration among team members. This strategy also accommodates unexpected absences and similar events well; the other developers working on that story can generally complete it.A key benefit of story swarming is that it avoids the problem of many stories being finished simultaneously at the end of the iteration. Instead, it provides a fairly continuous flow of stories that can be tested and accepted. There may be one or two stories that are finished at the very end of the iteration, but the rest have been completed earlier, giving QA and the Product Owner plenty of time to verify them. And since working software is the best indicator of progress we have, this approach can give a better picture of the iteration's progress than even a daily burn-down chart. This approach also resolves the danger of several stories being almost, but not quite, ready. If the team doesn't complete its work, it's generally only one or two stories that aren't done, so this approach allows a more acceptable failure mode. It's better to complete 75% of the stories, and have no progress on the remaining 25%, than to be 75% done with all the stories.It's also worth noting that there is little difference between this approach and the kanban approach to agile development . It is also compatible with iterationless approaches; story swarming accommodates very short iterations, and would accommodate variable length iterations easily, delivering one story per iteration. The strategy is not without its weaknesses, though. By focusing exclusively on a couple of stories at a time, a team might fail to address a risky item on another story in the iteration until it's too late. For this reason, a team might want to "cheat" a bit by having a developer identify and address risky items early in the iteration - for instance, prototyping a new feature and sending it to the Product Owner for feedback -rather than swarming a story with the others. After addressing those items, he can rejoin the rest of the team in working on their stories. This is a natural behavior under the "ad hoc" strategy that should not be forbidden by an overly-strict application of the story swarming rules. Story swarming is also vulnerable to a blocked story impeding the progress of several developers. Depending on the nature of the block, it's possible that the team could work around it for a while, and then leave one developer to continue trying to resolve it while the others worked on a new story.Finally, story swarming is slightly vulnerable to the problem of slackers spending all their time "pairing" or working on trivial tasks. However, since the set of available tasks is restricted to just the ones on the story being currently developed, it's much easier to detect and address this problem.
(The fore mentioned three strategies - Written by Robert Williams )

Strategy 4 - Team Leader/Tracker Recommendation
In the standup meeting, the technical lead (or team leader/tracker) who knows the capabilities of his/her team members can recommend (allocate optionally) to the team members the tasks of this day, and allow discussions with the team if there is any objection.
This strategy enables the team leader to resolve conflicts, develop the team technical skills, follow the plan of training & orientation more easily and provide more sustainable monitoring.
(Written by Ahmed Mahdy)

Recommendations
Story swarming is a good default choice for many teams, since it mitigates the weaknesses of other approaches without incurring any unique dangers that can't be worked around, and provides the unique benefit of a continual flow of stories. This approach is suitable for most teams in most contexts.Very mature teams may want to opt for the maximum flexibility and higher potential velocity offered by the ad hoc approach. These teams must ensure that team accountability maps directly to individual accountability, and establish a process to ensure that stories have enough time for testing and acceptance.Teams in very special circumstances may benefit from dedicating one developer per story, although this strategy has significant drawbacks. It may be useful for teams that can't collaborate as much due to geographic factors, and for teams that are very new to agile. Aside from those situations, the drawbacks associated with this strategy make it less appropriate than story swarming or ad hoc task selection.
(Written by Robert Williams )

What is Best, Scrum or Kanban?

I have for some time been thinking, what is best, Kanban or Scrum. I can't make up my mind so I decided to write an article in two parts, one where I wear the "I love Kanban" hat and one where I'm wearing an "I love Scrum" T-shirt.First I would like to tell you shortly what Scrum and Kanban is.Scrum in 1 minuteScrum is about getting back to the time when the company was small and everything was easy and ran smoothly. Back then projects were small, teams were small, releases were small and communication was easy. Best of all, we were efficient.In Scrum we split our big project into small projects as we work on timeboxed iterations called sprints. We split our big team into small teams (still with all the skills we need) and launch often. If we are more than 20 employees we probably have a problem knowing what the other departments and customers are needing so let's bring someone into the team that can represent them. To help communications from the team to the rest of the company we have our plan and current status visible. The plan is called the sprint backlog and the status is shown on the scrum board. Here is an example: The person we brought in to represent all stakeholders is called product owner and is responsible for prioritizing a list with features that will be developed for the product in the future. This list is called the product backlog.To make Scrum a process we add some small but necessary meetings: a planning meeting, daily synchronizing meeting and meetings to improve product and process. We decide the length of our iterations and aim to deliver something valuable after each iteration.

Kanban in 1 minuteScrum is simple but Kanban is even simpler. Just visualizing your workflow and limiting demand to capacity is enough to call it Kanban. Let's start by looking at the roots of Kanban. It's a way, invented and used by Toyota, to get a good flow of parts to their plants. Kanban is just a card attached to the part and when the part is used the Kanban is sent as a request to the supplier for more parts. The physical card is actually sent to the supplier who is attaching it to the next pert sent. Since the cards are re-used and no new cards are entered in the circulation Toyota limits the number of parts in stock and at the same time make sure new parts are ordered as soon as they are needed.

In software development Kanban is a limited promise of work. A Kanban board is a number of placeholders for work to be done. A typical Kanban board can look like this.

Each team has decided their limit and drawn as many squares as the limit allows them to. Whenever a task is done it will be moved to the next group if and only if they have free Kanban squares. Else the task is stuck. If there is nothing more to do we have a problem with the flow and we need to work together to solve what is stopping us. While Scrum is focusing on delivering on commitment Kanban is focusing on getting a smooth flow. In common for Scrum and KanbanThere are a lot of similarities between Scum and Kanban. Some examples are:
A strict prioritization (it's obvious what to do next)
Self-organizing teams (see below)
Transparency (show what you've done, your current status and what's next to do)
Inspect & adapt (improve your product and process based on facts)
Pull scheduling (see below)
Good communication and collaboration Self-organizing teams is when the team themselves are supposed to figure out how to solve problems, remove impediments and optimize their process. They should base their decisions on facts to inspect and adapt accordingly.Pull scheduling means the team is pulling work instead of having it pushed. This will limit work to capacity and lower stress, which is bad for both health and quality.This is why I prefer ScrumSo, now that you know what Kanban and Scrum is, it's time to decide which is best.
I start by putting the Scrum T-shirt on. Look further down if you like to read the part where I love Kanban.I concentrate on some areas where Scrum and Kanban differ:
Iterations - In Scrum you work in iterations. Kanban sees development as a forever ongoing flow of things to do.
Commitment - In Scrum a team commits to what they will do during a sprint.
Estimations - In Scrum you need to estimate to be able to have a velocity. In Kanban it's optional since focus is on time-to-market.
Cross-functional teams - That's one of the pillars of Scrum. For Kanban it's optional. Efficiency driven by commitmentBy asking a team for a commitment they will get a positive stress knowing that people from outside will see if they can do what they committed to. It also gives an energy boost at the end of the sprint when the team has the goal within reachable distance. After the sprint, the team can feel relaxed and gather some new energy for the next sprint. It's psychologically nice to not feel you are in the middle of a never-ending stream of things to do. Instead Scrum concentrates on one sprint at a time and allows you to focus only on those things you have decided to take on in the sprint. By working in fixed length iteration you get a nice rhythm the whole company can feel and adapt to.The value of estimatingEstimating is not a waste of time since valuable information is transferred from the product owner while the team asks what they need to know to be able to estimate. The same thing applies to commitment. To be able to commit team members need to know what they are committing to. They simply have more interest in getting information from the product owner so they listen more carefully. And they are just asking for the information they need which makes the meeting very efficient.Even though you can predict when a project is done, just by counting stories left, it is more precise if each story is estimated as well. Prediction is important when more departments, like marketing, are dependent on your work. It gives a more serious feeling.Slack at end of sprintsIf there are some time at the end of a sprint that's valuable even if the time is not big enough to start developing new features. This is time developers can use to improve code quality or the development environment. Both will give better efficiency in the long run.Build teams with cross-functionalityWorking together in a cross-functional team is a psychological thing. You are able to go the whole way from concept to released product together as a team without dependencies. You are not just a cog in the machinery. It's also fun and helps you grow as a person. By helping each other, knowledge is spread which lessens dependencies on each person. It also helps prioritize since features can be built without considering which knowledge is available. To coordinate between people with the same skills who are sitting in different teams you can have virtual team meetings as often as needed.Scrum is best for beginnersI know, you can do all those things even when doing Kanban but they are optional and since they are optional you might just not do them just because you are too lazy to challenge the usual procedures or structures of your company. If you are new to Agile development it's better to have a method guiding you to do the right things. Just read the book and implement it. Later when you get used to it you can start experimenting and eventually get better performance. If you start experimenting too early you might end up with waterfall or chaos but still calling it Agile. Kanban is more advanced because you need to make a lot of decisions without having many facts.This is why I prefer KanbanNow it's time to take off the Scrum-T-shirt and put on my "I love Kanban"-hat.
Kanban is based on Lean principlesKanban is Lean and has been around for 50 years and has shown to be successful. Things are seen as a flow without iterations. Not many rules. It's just a focus on reducing work in progress, strict prioritization and limiting demand after capacity. Besides that you have the normal Lean principles of:
Quality
Just-in-time (decisions and facts just when they are needed)
Short lead-time (quickly from concept to cash),
Kaizen (continuous improvement)
Minimizing waste (everything that is not adding value to the customer) No meetings if it's not adding value to the customer. Most of those things are fine with Scrum but I can sometimes think Scrum has some waste. Here I will mention some areas where Scrum practices might be waste.Estimations might be wasteIf a product owner knows what needs to be done and the only goal is to have it done as soon as possible, why spend time on estimating? In Scrum, estimations are needed for a team to know how much to commit, to calculate velocity and for the product owner to decide on prioritization. If none of this is needed, it's waste and we shouldn't do it. If you like to measure velocity you can instead count the number of stories done per week or month. While talking about measurement why not look at something the customers are interested in, lead-time. That is, the time it takes for the customer to get what he or she ordered.Constant flow If the team is done with a story and there is not enough time left in the sprint to complete the next, it's likely no one works hard the remaining time and that's waste. Kanban is trying to get a steady and non-stressed but high flow of work running through development.Sitting with people that can help youIn Scrum a team is cross-functional and sits together with the people working with the same project and not with the ones with the same knowledge. Let's say you work with sound effects for a computer game. Then it's not much help to have a java-programmer beside you. It would be more valuable to sit with people doing the same things as you do who can help you with tools and practices.

No possibility to commitIn Scrum, the team commits which does not work very well for a team with support issues. They get a lot of disturbance and commitment is not worth anything since the ones disturbing them don't care about commitment. A commitment you can't control gives either frustration and stress or demotivation.Kanban for support and operation teamsSupport and operation teams are often interrupted. It's part of their job. Frustration comes when interruptions are happening just because there were no time for pre-emptive work. Just by having the pre-emptive work on the board gives focus to fix the hardest problems one-by-one, which gives less interruptions and more time to work pre-emptively. It becomes a good circle. A Kanban board for an operation team can look like this:

Most important are the urgent problems that arise. Secondly, daily duties must be done. Then the team members can take care of other projects, usually to improve stability in the operation environment. The limitation on the number of parallel tasks gives focus to be done working on improvements, not just starting them.Many options gives freedomKanban is simple with few constraints, which gives you freedom to change your process step by step. Either you go from waterfall or chaos, your first step is to visualize and limit your flow with a Kanban board. When you have your flow visualized you add whatever work is needed to thrill your customers. Your improvements might be decided during a monthly retrospective meeting or on a daily meeting. ConclusionI still can't make up my mind if I like Kanban or Scrum the most. My conclusion is, not very surprisingly, that it depends on the situation. See them as two different tools and find out which tool is the best for your situation. Maybe you start with Kanban and end up with Scrum or maybe you don't. Maybe you'll start with Scrum but change to something more like Kanban. Whatever you do, just make sure you do what is best for your customers.

Ref:

http://www.agilejournal.com/articles/17-articles/1737-what-is-best-scrum-or-kanban

Group Coherence for Project Teams - Practice for Continuous Improvement

One of the principles in the Agile Manifesto states, "At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly." This principle guides both aspiring and seasoned Agile teams in the pursuit of continuous improvement and can support whatever Agile adoption path an organization may choose. Kent Beck adds observations about this topic.
"'Continuous' improvement is a bit of a misnomer. It means continuous awareness, responsiveness to feedback, and openness to improvement. When you know how to improve, then you improve. You make a change; observe the effects; then digest the change, turning it into a solid habit. Eventually you hit a plateau from which you can absorb more feedback and identify the next opportunity." (Beck & Andres 2005, p141)Extreme Programming Explained: Embrace ChangeAddison-Wesley Boston MAThis quote describes a process that we recognize as Practice, defined in our previous article, Group Coherence for Project Teams - Common Purpose, as "The ability to identify difference and learn through attentive repetition, either for individuals or groups... Through Practice we can get a feel for the contextual application of knowledge, rather than the acquisition of individual techniques." Practice, an ingredient of Group Coherence, is the only ingredient that the research group found to be connected to every other ingredient. Beck refers to continuous improvement in the context of an individual's learning process. "Change myself, then offer the fruits of that change to others." (p141) Practice is a process through which groups can also learn and change.
Although individuals contribute to the features developed by a project team, the product of their work (the features themselves) does not belong to an individual, but is the product of the entire group. Just as the music produced by a jazz ensemble includes the individual improvisations of each member, the music belongs to the group. To whom does the technical debt and code quality in software systems belong? Why do some environments not take advantage of the opportunity for iteration provided by Agile methods to turn releases into non-events?We propose that items such as code quality, technical debt and release cycles have tacit ownership at the group level. This is unconscious, as group ownership is often minimally recognized in organizations. These unacknowledged group items are vulnerable to decisions and prioritization made by individuals without the group's voice, at the expense of the opportunity for the group to learn. Untapped group learning can be brought to awareness through group Practice thus supporting the pursuit of continuous improvement in these often challenging areas.Challenges - Release ProcessLarge organizations are prone to centralize their release process in an attempt to protect their live systems from rogue updates, or their customers from a greater pace of change than they can absorb. A common implementation of this protection is to use an additional team for production releases independent of the development team. In reality the development team is invariably called in to resolve issues that surface during the release process. This illustrates tacit ownership of the release process that belongs to both release and project teams. The whole-group ownership of the release is not acknowledged. The resulting integration feedback loop is late, long and cumbersome and releases are big events involving multiple departments and stressful interactions with long hours.In environments where a centralized release process is assumed to be the constraint and not up for discussion, Agile is stunted in its ability to provide continuous Practice in the software delivery. A common reason to avoid the process conversation is the excuse that "our complex business needs such a process." However, by not allowing Agile teams to release to production early and often, we prevent the Practice that would allow them to turn the release process into a non-event. Challenges - Technical DebtThe centralized release process implementation can result in development and test environments with limited ability to test performance, integration, code coverage or product usability. This forces a lot of critical testing to be done during the release rather than the development process. The project team's best efforts to deliver a "safe" and tested feature set become little more than guesses. The developers have to attempt all their testing in the System Test environment. This is like developing blind as this environment lacks the connections to the system's external integrations.The project team's group product consists of all the features and all the tests. Centralizing the release process fragments group ownership. Developers focus on unit and system tests and don't have access to accomplish the full set of critical tests. The release team can focus on the tests the developers can't execute but release engineers do not have access to the code. The result surfaces technical debt that neither team is empowered to own. When responsibilities for the group product are fragmented, no whole group exists to own that product. Practice by the whole ownership group could improve both the release process and catch technical debt earlier.Challenges - Code QualityCustomers and product owners may press the project team for a wide feature set and an aggressive delivery date. This can compromise code quality, causing the developers to express concern. The two groups might have a different interpretation of the Agile principle for "minimizing the work not done". Customers might think that refactoring is unnecessary waste since this effort results in no visible changes to the application. They often challenge the value of what they might describe as "writing the code twice". However, once a feature is implemented, an Agile development team refactors the code to make it more efficient to run, cheaper to maintain and easier to extend. So who does the code quality belong to? The customers are paying for the work but the developers have a fiduciary duty to ensure that the customer receives maximized value. The code quality is part of the whole-group product. This disagreement shows that in this case the true communication never happened. The need is to resolve the disagreement and create collaboration.Code quality is built into the software over time. If the opportunity for collaboration is limited to the final stages of release preparation, the ability to identify deficiencies in code quality is also limited. An occasional look at the system does not provide the opportunity to develop trust, align goals and expectations, and learn to communicate effectively as a team. Practice to Enable Whole-Group OwnershipWe would like to share our personal experience of one way in which we have turned the tacit group ownership of release process, technical debt and code quality into a conscious seamless part of the day-to-day interactions of a whole project team. In a financial risk management system like the one in this example, projects waited until a complete financial model was implemented in C. Once the traders validated this model, the developers could begin to extend their risk management system to use it. After this software development was complete, operations and support processes could be created. These sequential handoffs were an established project life cycle.When market conditions suddenly invalidated an existing financial model, a business need arose to extend the risk management system considerably without knowing what replacement model to use. Using an invalid model put traders at great financial risk and under considerable stress to manually adjust every quoted price. This was a crisis that could not wait for a sequence of handoffs over the next 12-18 months.Instead, by initiating communication with all the stakeholder groups simultaneously, it became clear that each had a part of the process and also a part of the solution. To make the communication among the stakeholders both practical and concrete, developers created sandbox versions of production for each conversation among traders, financial modelers, trading administrators and support engineers. The increased level and quality of communication led to the realization that the whole group owned the system.Implementing the sandboxes removed the handoffs and enabled the participation of the whole team throughout the project. The whole group was able to take ownership of the technical debt, code quality and release process, since each of these sandboxes made these items visible to everyone involved. Most sandboxes were set to refresh their database daily from the live system, and would be extended with all the daily code changes from the development system as well. In addition, the financial modelers were evolving multiple analytic libraries, and the latest versions of their work were also applied to the sandboxes on a daily basis.Traders and their administrative staff picked up any drop in code quality on the part of either the developers or the financial modelers immediately. Any technical debt that impacted the usability of the system or the daily generation of risk or profit and loss figures was also picked up immediately. Degradation in the computational performance of the application was obvious to everyone straight away. And of course, the release process was being tested on each sandbox every day. An Environment for Group LearningThe previous example illustrates how software development can be a group learning process. This solution allowed for both Practice by the entire group and also for continuous improvement in several areas. For example, the financial modelers were able to share each other's work and thus were able to explore four or five times the normal number of candidate models to develop a solution, with each cycle of modeling providing input for continuous improvement. The traders were able to test numerous hypotheses about the market parameters that would influence the new models, thereby contributing improvement to their business model and trading strategy.Given the crisis, the stakeholders were willing to do away with the traditional handoffs, include all the participants and create a shared sense of common purpose. This holistic project interaction initiated group Practice in which self-organization and group learning spontaneously emerged. When somebody suggested something that the group could do, the group provided a way for them to try it, improving the group's environment with each cycle. Eventually, the group was able to create its ideal environment for continuous improvement. This environment was the result of acknowledging the previously tacit group ownership of the risk management system. The Practice allowed a new self-organization to emerge. ConclusionWhen everyone is able to see and experience the progress made throughout the development process, they will offer and receive feedback without being solicited or prepared. This open group exchange is one result of Practice. To guide a collection of Agile practitioners and their project stakeholders towards Group Coherence, it is important to provide opportunities for the group to Practice ‘being a group'.Without ‘being a group' the team has no access to the tacit knowledge. Once it accesses its shared understanding, the team develops a knack for group learning. Group learning is necessary for a team to embrace suggested improvements. Continuous improvement depends on group learning provided through Practice.Guiding a group from Practice to continuous improvement can allow practical results in such challenging areas as code quality and technical debt and can make releases so frequent, tested and drama-free that they become invisible.


Ref: http://www.agilejournal.com/articles/17-articles/1742-group-coherence-for-project-teams-practice-for-continuous-improvement

Agile Automation: Automating The Agile Release Process

Agile methods promote a process that encourages frequent inspection and adaptation, the rapid delivery of high-quality software, and responding to changing circumstances rather than strictly following a plan. Therefore, automation is key for the success of the Agile release process.
To make sure Agile projects run smoothly and rapidly, companies must automate all repetitive activities – such as installing new builds of complex, multi-tier, multi-server software. Automating application updates in development and QA environments accelerates the release process, increases flexibility and efficiency, and enables companies to keep the Agile promise of efficient, rapid delivery of high-quality software.
In an Agile environment, software requirements are constantly changing. The Agile promise of ensuring customer satisfaction by rapid, continuous delivery of useful software means that software is delivered in less time, but each released build must still be fully tested. It’s obvious that in this type of environment, manual processes will fail. Manual processes can only work if your product is easy to deploy, such as a desktop application deployed on a single computer.
Automation of the Agile release process by Nolio enables companies to automate the deployment of testing labs and the updating of development environments several times a day. This allows developers and testers to always work with the latest build, even in complex application environments, which are constantly changing.
In the rapidly changing environment of the Agile release process, your business simply cannot afford slowing down or compromising product quality because of human errors, failures and downtime. Fully automating the Agile release process is the answer.
—Nolio Application Service Automation is a software platform for designing and executing automated application service workflows across the data center, enabling reliable, effective processes for the management of application change.

Process improvement meets 'Peopleware'

People are the key asset of any organization

by MARY SAKRY

If you have heard Timothy Lister speak, you can better understand my enthusiasm for his work. This issue I would like to talk about how important "Peopleware" concepts can be to your process improvement efforts.

This article will quote heavily from authors Lister and Tom DeMarco. All quotations are straight from "Peopleware."

Determinants of process improvement success

When we think of process improvement, we must consider the balance of three components: process, people, and tools. All three of these components play an important role in determining how successful the process improvement results are.

We need good processes to accomplish our software development and maintenance with the appropriate tools to support these processes while realizing that it is people that do the work and use those processes, methods and tools. Therefore, ultimately people are responsible for the success or failure of the improvement effort.

Lister knows how to drive home the importance of the people who create software better than anyone else. He always maintains a level of wit that makes his important message palatable yet ensures that his message hits the mark.

As you work on your process improvement issues it is important to keep this element in mind. Most SEI Level 12 findings are behavioral in nature, and organizations can do well by paying close attention to his message.

"Peopleware" in a nutshell

Here I state several points made in "Peopleware." My intent is to tell you just enough so that you will read the book yourself. So I won't give much detail.

The major problems of our work are not so much technological as sociological in nature.

"We're all under a lot of pressure to improve productivity. The problem is no longer susceptible to easy solutions, because all the easy solutions were thought of and applied long ago. Yet some organizations are doing a lot better than others." Several helpful ideas to improve productivity are offered.

The authors examine the office environment and what can be done to enable higher productivity.

"The manager's function is not to make people work but to make it possible for people to work."

The book tells us how to: "get the right people, make them happy so they don't want to leave, and then turn them loose."

What's in the foreground of most of our prized work memories is team interaction. When a group of people fuse into a meaningful whole, the entire character of the work changes."

"In the best work groups, the ones in which people have the most fun and perform at their upper limits, team interactions are everything. They are the reason that people stick it out, put their all into work [and] overcome enormous obstacles." Lister and DeMarco present some good ideas on how to grow and maintain productive teams.

Finally they leave us with the notion that work should be fun. "We need people to recognize the occasions when they are being asked to do something silly -- and actually say so. Once it's been said out loud, (the organization) can't ignore it any longer."

What does all this mean in relation to process improvement? And why should you care?

First of all, some of you may have had a process assessment, and if you have, you may have been surprised that so many people-oriented issues surfaced. If you have not used an assessment as the basis of your process improvement, then you should be advised that you may have missed some critical issues that are people-related.

No matter how you have set up your improvement program, you must deal with these issues as they are foundations for the improvement work to follow. For example: If people are expected to complete projects based on unrealistic schedules estimates or work in an environment that allows continual interruptions, then to think that they will be able to make meaningful process improvements may be unreasonable.

As a rule, people enjoy working on process improvement as long as it is not seen as an additional burden to an already hectic schedule. When people are dedicated to process improvement, it should be considered part of their job and time must be allocated for the effort.

Keep the software people happy

Be sure to keep a balanced emphasis on people, process, and technology.

If, during your efforts, you are unable to help people solve their real problems or are unable to convince them that you can help them solve their real problems, your effort will fail. Remember, the software professionals are your customers and you want to keep them happy.

Agile Methods and Virtual Distributed Teams

By Dr. David F. Rico, PMP, CSM

Are Agile Methods and virtual distributed teams compatible? Aren't Agile Methods and virtual distributed teams polar opposites? Aren't they incompatible? And, if they are compatible, what are the specific techniques for making Agile Methods work for virtual distributed teams?
What does all of this mean? What is this controversy surrounding Agile Methods and virtual distributed teams? Why wouldn't Agile Methods be compatible with virtual distributed teams?
Well, all of this is related to the principles and values of Agile Methods as defined by the Agile Manifesto (http://www.agilemanifesto.org/). The sixth of 12 major principles is as follows, "The most efficient and effective method of conveying information to and within a development team is face-to-face conversation." It relates to two of the four major values of Agile Methods: (1) individuals and interactions and (2) customer collaboration. So, taken at face-value, no pun intended, one would be omitting half or 50% of the values of Agile Methods if one didn't use face-to-face conversations with customers and team members. Or, would they?
Face-to-face communication is contextually-rich, and is the preferred method of communication in Agile Methods. In Traditional Methods, statements-of-work, requests for proposals, contracts, requirements, and designs are considered contextually rich. But, nothing beats good old face-to-face communication when it comes to interacting with customers and team members. The creators of Agile Methods realized this and etched face-to-face communication in stone as one of their 12 commandments.
Okay, now, so anyone who can read a book on Agile Methods knows that face-to-face practices are mandatory. That is, anyone except those who are trying to make money in the global software industry. The global software industry is all about connecting or automating the supply chain of goods and services using products from China, computer programming labor from India, project managers from North America, and researchers in Europe. Well, that's simple, just co-locate everyone. Not so fast, why can't everyone stay exactly where they are and use electronic means to communicate? Well, that would involve violating the principles and values of Agile Methods, wouldn't it? In fact, if a team is not face-to-face, it's using Traditional Methods, not Agile Methods. Or, is it?
The creators of Agile Methods had part of the equation correct. Face-to-face communication is contextually rich, and is therefore superior to virtual distributed communication or software documentation in certain circumstances. However, the North American creators of Agile Methods failed to anticipate or acknowledge that 95% of the world wasn't collocated with them, and that North America wasn't even the predominant computer programming market. Didn't they heed the warnings of Ed Yourdon who said that the Indian software market was skyrocketing or how about Michael Cusumano's warning that Japan's Software Factories were taking over the planet? Perhaps they were caught up in the euphoria of Microsoft's and Intel's instant success in the 1990s. Or, the Internet Gold Rush convinced them that North American information technology dominance was here to stay?
I think the only ones who didn't believe the creators of Agile Methods were the 70 or so other countries besides the United States, who benefited from the Personal Computer and Internet revolution as well (along with any enterprising capitalist who wanted to make a quick-buck interconnecting the global supply chain).
The problem is that 99% of the population who knows anything about Agile Methods believes that one isn't being Agile if one isn't using face-to-face communication and all of the associated trappings (e.g., onsite customers, pair programmers, daily standup meetings, etc.).
Well, hang on to your seats, because the world is changing as we speak, and virtual distributed teams for Agile Methods are here to stay. We all have the benefit of being on the leading edge of this phenomenon as part of the next phase of software engineering evolution. I bet most people who are delving into Agile Methods didn't even realize they were on the cutting-edge of software engineering when they endeavored to use Agile Methods? Let's take a look at some of the evidence.
The folks at British Telecom have devised a set of practices for making Agile Methods work with virtual distributed teams (Cannizzo, Marcionetti, & Moser, 2008). They've identified four major practices to help marry Agile Methods with virtual distributed teams: (1) maximize project status visibility to all stakeholders, (2) ruthlessly automate as many development and management processes as possible, (3) ensure effective communications to the maximum extent possible, and (4) provide immediate feedback on every task performed. They use Eclipse IDE for writing code, Fitnesse and Selenium for acceptance testing, CruiseControl with Ant as a build tool, Danube ScrumWorks for user stories, Subversion for version control, and Atlassian Confluence Wiki for document sharing. Other tools include Visual Studio, NetBeans, Capistrano, Live Meeting, Communicator, Meet Me, and a variety of team building tools.
The folks at Yahoo! (Drummond & Unson, 2008) suggest mandatory meetings within time zones, periodic face-to-face meetings across time zones, overlapping virtual distributed meetings where possible, periodic synchronization between international Scrum Masters, the use of Wikis to share photographs of user stories on Post It notes, and ensuring that virtual distributed teams maintain the rank-ordering or priority of user stories. Furthermore, Yahoo! says to emphasize individuals and interactions over processes and tools as much as humanly possible, factor in the needs of the global workforce over individual practices of Agile Methods, and ensure that accurate information is shared in a timely fashion.
The folks in Canada (Robarts, 2008) say to have periodic face-to-face meetings as much as possible (especially during project kickoff), periodically send people back and forth from one country or location to another, ensure domain experts periodically visit one another, ensure delivery teams occasionally visit one another face-to-face, hold virtual distributed daily standup meetings between people in the same time zone, exchange informal notes such as PowerPoint presentations as meeting minutes, and ensure Wikis and other automated tool content is constantly up-to-date. The Canadians remind us to emphasize the use of Agile Methods practices like User Stories (instead of specifications), use video conferences as much as possible, and use portable information technologies such as laptops, cell phones, and personal digital assistants (e.g., BlackBerries). More importantly, proper schedule management is of utmost importance, especially when it comes to building in management reserve, accounting for international Holidays and unplanned events, and keeping the schedules current as well as communicating them. Interestingly enough, we're advised to allow enough latitude for differences in individual Agile Method practices across international boundaries.
The list goes on and on, as the Indians recommend automated dashboards and wikis (Shrinivasavadhani & Panicker, 2008). The Americans emphasize WebEx Meetings, periodic virtual distributed meetings between international Scrum Masters, and a variety of techniques such as balancing power, allowing for variation in practices, detailed User Stories, empowerment, sharing vision statements, executive commitment, and as much communication as possible (Therrien, 2008). Another Canadian team says to use nearshore resources, maintain strict communication plans, share electronic workspaces, use synchronous communications only in adjacent time zones, and use asynchronous retrospectives (Vax & Michaud, 2008). Finally, a group of Americans and Canadians recommend instant messaging, synchronous communication mechanisms, video conferencing, face-to-face virtual distributed meetings, virtual distributed standup meetings, and the list goes on and on (Young & Terashima, 2008). You don't want to skip this last paper, because it contained a lot of helpful advice.
So, what's the bottom line? The creators of Agile Methods had it right, face-to-face communication is contextually rich and face-to-face is the preferred method of communication. And, a lot of face-to-face communication is a key, if not the key, to project success. I speak from experience on this one, because one of the most difficult projects I've ever managed, was success due to frequent, unscheduled face-to-face communication to break down the barriers to personal trust. Once the barriers to trust had been breached through frequent personal interaction, then the project completed successfully. We were given an award by a customer who had never given an award to a consulting firm before. You just don't find that kind of advice in your typical textbook on Traditional Methods. So, how do we duplicate the benefits of face-to-face interaction in virtual distributed teams? Communicate, frequently and often! Outside of video teleconferencing, telephone calls have been one of the best forms of communication over the last century. It's the "unscheduled" telephone calls that provide the most return-on-investment (versus regularly scheduled telephone calls that just don’t seem to have the same effects). Ad hoc telephone calls help alleviate anxiety instantly, as opposed to scheduled phone calls. Intimate, personal communications break down barriers to trust and ensure project success almost every time.

REFERENCES

Cannizzo, F., Marcionetti, G., & Moser, P. (2008). Evolution of the tools and practices of a large distributed agile team. Proceedings of the Agile Conference (Agile 2008), Toronto, Canada, 513-518.

Drummond, B. S., & Unson, J. F. (2008). Yahoo distributed agile: Notes from the world over. Proceedings of the Agile Conference (Agile 2008), Toronto, Canada, 315-321.

Robarts, J. M. (2008). Practical considerations for distributed agile projects. Proceedings of the Agile Conference (Agile 2008), Toronto, Canada, 327-332.

Shrinivasavadhani, J., & Panicker, V. (2008). Remote mentoring a distributed agile team. Proceedings of the Agile Conference (Agile 2008), Toronto, Canada, 322-326.

Therrien, E. (2008). Overcoming the challenges of building a distributed agile organization. Proceedings of the Agile Conference (Agile 2008), Toronto, Canada, 368-372.

Vax, M., & Michaud, S. (2008). Distributed agile: Growing a practice together. Proceedings of the Agile Conference (Agile 2008), Toronto, Canada, 310-314.

Young, C., & Terashima, H. (2008). How did we adapt agile processes to our distributed development? Proceedings of the Agile Conference (Agile 2008), Toronto, Canada, 304-309.

Product Backlog Rules of Thumb

While working with Product Owners over the years I have learned some rules of thumb that help make their Product Backlogs more manageable. Some of these rules of thumb I learned from other people and some were learned through trial and error.
Remember that the statements contained in this article are just rules of thumb intended to help guide your management of a Product Backlog. They are not rules to adhere to no matter what. Always use common sense when applying a rule of thumb to your particular context.The first rule of thumb that I would like to discuss is the:
Keep your Product Backlog between 50 and 100 items total
It is common for a team or organization to implement Scrum and find their Product Owner overwhelmed by the mass of the Product Backlog. The Product Backlog is filled with every feature request, suggestion, defect and infrastructure improvement. The Product Owner is no longer able to effectively manage the Product Backlog and maximize the value of delivery through prioritization.To combat this unmanageable mess, gather related Product Backlog items into a single, larger Product Backlog item so they can be managed as a group. As these combined Product Backlog items get higher in priority they can be easily expanded into their smaller, contained items and prioritized individually among a smaller subset of high priority items. Also, go through the Product Backlog and remove items that are not directly aligned to the product roadmap. Frequently, items are kept in a Product Backlog because they sound like a good idea but are not aligned with the overall product strategy. Figure out if these items should be represented on the product roadmap or are they a distraction from the product goals.Next, lets make sure the Product Backlog is ready to conduct effective Sprint planning with by making sure...
The top 20 to 40 items are small enough to fit into the development team's Sprint
The top of your Product Backlog represents the highest priority items to be implemented by the development team. It is important to work with the team on making these items small enough to fit into an upcoming Sprint. Identifying the appropriate size of the top 20 to 40 items involves getting estimates of cost from the team. From these estimates, the Product Owner can gauge the appropriate size of items to fit into a Sprint. If the Product Owner believes a Product Backlog item is important but sized too large to fit into a Sprint then the team can help the Product Owner split it up into smaller chunks or simplify the implementation.An appropriate size could be described by the following formula...
The upcoming Sprint should fit at least 4 of the highest priority Product Backlog items
After a team has executed 1 or more Sprints they can calculate their velocity based on data from those Sprints. Velocity is the amount of Product Backlog delivered within a single Sprint. Calculating the velocity is a function of Sprint duration, team makeup, sizing nomenclature, and product technology. Therefore if any part of that function changes then the velocity is no longer valid. The best way to get predictability in team delivery is...
Keep the development team together working on the product with a continuous Sprint cadence
The rhythm created with each Sprint having the same duration will help the development team understand how much they can commit to in planning and deliver successfully as potentially shippable code by the review meeting.
Have a description of the Product Backlog item is usually not enough to make it clear for estimating by the development team. It is helpful to...
Put acceptance criteria on top 20-40 items
Acceptance criteria could be a bulleted list of capabilities the software should exhibit once the Product Backlog item has been implemented. These can be thought of as the functional acceptance tests for the Product Backlog item. The development could also identify technical aspects of the functionality as acceptance criteria with acknowledgment of the Product Owner.
If your list of acceptance criteria is getting larger than 7 then...
Break up the Product Backlog item into multiple items partitioned by common functionality within the original item.
Of course, there are also times where the Product Backlog item is too small and does not do enough on its own to create value for the product. In that case there is the following rule of thumb...
Each Product Backlog item should have at least 3 acceptance criteria
Having Product Backlog items that are too small is not always a bad thing but it could be difficult to manage and provide visibility into. I have found it also leads to Product Backlog items that are technical in nature without direct business benefit. The Product Owner could find it difficult to gauge the correct priority for an item that is too technical so...
Every Product Backlog item should present value that a Product Owner can understand and prioritize effectively
Abuse Stories can be used to drive out the value to users for technical infrastructure or architecture Product Backlog items. An Abuse Story identifies the cost of not addressing a technical aspect of the product. For instance, in the case of fraud, an Abuse Story such as the following could be identified as a Product Backlog item:
As a Malicious Hacker I want to gain access to credit card information so that I can make fraudulent charges
The acceptance criteria for this Product Backlog item would be the technical modifications that will be made to stop the Malicious Hacker from gaining access to credit card information. The risk identified in this Product Backlog item helps the Product Owner better prioritize beside other potential candidates for upcoming Sprints.
The entire Product Backlog should be prioritized in stack rank order
And...
The development team that is going to implement the product should estimate all Product Backlog items
These are the basics for Product Backlog management but I thought I should reiterate them. I often find that the Product Owner does not take time to prioritize their entire Product Backlog. If kept to a manageable size, the Product Backlog provides visibility to stakeholders, management, and the development team about what is coming in the near future. It is difficult to prioritize if the Product Owner does not have a cost to weigh against the benefit for each Product Backlog item. The team who is actually going to implement items off the Product Backlog should be the only people who estimate the items. No individual outside the group should tell the team what the estimates are. After the Product Backlog is fully estimated, the Product Owner should...
Generate a release burndown chart to monitor progress toward a release of the product.
Once you know how much the team can deliver each Sprint (velocity), you can do some ad-hoc release planning by going through your estimated Product Backlog and finding out how far they will be after n number of Sprints. This ad-hoc plan can be monitored on a release burndown chart to see if any changes must be made based on the reality of current implementation or emergent requirements.This listing of Product Backlog rules of thumb is just a start. I have found it helpful to go through this list with new Product Owners or Product Owners finding it difficult to manage their Product Backlog. As a Product Owner becomes more proficient in their Product Backlog management technique then they can step away from some of these rules of thumb to optimize for their environment.

A Critical Look at CMM and Agile Through Gen Y

Much has been written about the commonality or even the lack of it between Agile and CMM [1,2,3,4]. CMM claims to be a flexible model that can be tailored and adapted to many life cycles [4,5,6]. This facilitated attempts to bring-in the iterative life cycles and practices of Agile under CMM model [1,2,3,4]. However, the debate on their compatibility continued as is evident in Kevin Trethewey‘s blog, which is one of many of its kinds[7].The continued discussion is evident enough that it is not sufficient to compare practice to practice and arrive at any reasonable conclusion about their overall compatibility.The one difference that seems to have earned consensus amongst most of the published literature on the subject is "the people aspect" of these models [3,4,8]. While CMM has little people focus, Agile's foundation is people focus. In the context of Generation Y becoming a major part of the work force, this people aspect may become one of the main drivers for organizations to endorse Agile and reject CMM with or without Agile practices in it [9,10].2. People Focus in CMMThere are multiple aspects of CMM and its implementation makes the people aspect take a back seat. As James Bach puts it "The CMM reveres process, but ignores people." [8]Following is a compilation of some of these aspects culled from the available literature..2.1 Historical baggageHistorically, CMMi's predecessor CMM was originally published in 1989. This date is much before the advent of Internet. Following is an excerpt from a relatively recent paper from SEI suggesting to embrace both Agile and CMM"If we look at the genesis of the CMM, it predates the internet and nearly everything associated with internet technology. For that matter, CMM predates many software development, deployment, and infrastructure technologies, languages and methods. We've all learned a lot in the past 20 years. When the DoD set out to address their "software dilemma," the software world was different than it is today." [4]CMM was mainly focused on models for high-risk, government contractor projects. In the age where there was not so much communication let alone collaboration channels established, the model had to be something that enforced accountability rather than expected it.CMM later expanded to multiple other models and the latest being CMMi 1.2. However, the remnants of old times are present even in today's models. The following reference states that explicitly."While the language of CMMI, admittedly, may retain some of the flavor and phrasing of this context, each release of a CMMI model grows further away from these roots to embrace a richer and more dynamic set of contexts." [4]The same paper goes on to explain how the situation worsens further by practitioners who don't move with newer versions of CMM that are made to suit to modern contexts. All this makes CMM and its newer variations a distant old relative to the current generation. With little effort anyone can relate to their distant older relatives, but it would be inconceivable that they would be convinced to live like the earlier generation.Defined with the value system having roots in a contractually handled market system, all the relationships would be between non-life abstract entities like organizations but not people."CMMI for Development, Version 1.2". [6] recognizes people amongst the three critical dimensions of an organization. But it emphasizes that it is the processes that hold the other two. Since the technologies are changing quickly and people are regularly changing jobs, CMMi believes to entrust more on processes that can be made static and can address the other two dynamic dimensions. Following is an excerpt that draws an analogy with manufacturing industry."Manufacturing has long recognized the importance of process effectiveness and efficiency. Today, many organizations in manufacturing and service industries recognize the importance of quality processes. Process helps an organization's workforce meet business objectives by helping them work smarter, not harder, and with improved consistency."[6]The historical fears of the changing world may not be relevant today as the s/w is not organized; either in punch cards or long procedural codes. The object oriented, componentized, building block style of code with the very advanced tools does not make changing people and technologies as big an issue as it was in earlier decades.The analogy with manufacturing to s/w industry is rather demeaning. In the industrialized world of the manufacturing industry, work is done and goods are produced by machines. Humans only operate machines that are part of a long assembly chain. On the contrary, if it is humans that have to deliver goods that too of intellectual nature, the analogy with manufacturing industry cannot withstand. The only analogy that can suit the situation would be creation of art [20].2.2 Consulting business modelIt has been mostly the services/consulting companies that went for appraisals and certifications. This is the main reason why there are more Indian IT companies with more certifications than others [11,12]. It would be difficult to find product/technology development companies like Oracle, Apple, Google going for such elaborate process models and certifications. The distinction is that the technology-centric companies need not prove their ability to build new technologies to the external world. The end product in the market place establishes their capability. When it comes to consulting companies, they cannot exhibit "a product" in the market place; since service is a matter of customer experience. Earlier client references may fulfill a part need to showcase their ability. However, a need for a more credible body certifying a consultant company stays intact.Following excerpt from the case study "Managing Growth in a High Technology Venture - the Fact-Tree Way" explains this need.""Processes" and "Quality Certifications" were the buzz words now. It was not just the Fact-Tree management that felt a strong need to get the organization certified, the prospective customers, especially the European clients, preferred firms that could flash a Quality certification such as ISO." [9]The firm discussed in this case study starts with an ISO certification and then suddenly finds itself in the need for an SEI-CMM assessment. The reasons for this as stated in the paper are not necessarily to do a better job, but sustainance in the competition with other companies in the certification race.The decision of these companies going for ISO, CMM or an Agile model tomorrow is not because of their own convictions and/or internal needs; but would be determined by their competition and what is a "hot" certificate in the market place."Meanwhile, Fact-Tree's top competitors were going for SEI-CMM (Software Engineering Institute- Capability Maturity Model) certification and Fact-Tree did not want to stay behind." [9]It would be an easy observation for anyone who works closely with Indian IT companies. In such an environment where flashing a certificate becomes the business need, everything else including people focus takes back seat. To start and survive in such market starts with the strategy of how quickly the certification can be achieved.As Nagaraju puts it in his reply to the case study "What is wrong with the Indian Software Industry?" [10], this environment creates more value for the people who fall in line with the process and make them grow up in the career as leaders and managers."The CMMI does not preclude innovation and change. The process owners are the ones who are the gatekeepers to progress." [13]The core skills needed to perform a technical job becomes secondary. These incapable managers and leaders who grew up quickly not because of their technical abilities will find it difficult to work with the programmers working for them. This in turn results in the mistrust between managers and teams.Quality groups, checklists and audits become the major activities in such organizations."The job of the quality department was felt to be policing rather than playing an important role in developing the system." [9]The mistrust in the atmosphere results in manufacturing data, last minute documentation and an overall unhealthy atmosphere."The perennial problem is that of employees not being too keen on the processes and treating processes as a burden. New entrants still feel that documentation is a hell of work and the tendency for last minute documentation is a common phenomenon."[9]3. People Focus In AgileAgile by definition is a bottom-up approach. Team gets to decide what works for them. There is an added trust and responsibility on the team to retrospect and fix its own course... and most importantly this is not driven from the top or external departments. Trust leads to responsibility that leads to commitment. This contrast of "we control our course" vs "we follow and work under a prescriptive process/model" becomes very significant differentiator.3.1 TrustThe very first line in the Agile Manifesto states "Individuals and interactions over processes and tools"[14]. This being the major value definition, any and all the flavors of agile keeps this at the top of their priorities."Scrum believes that people, their skills and how they are harnessed are the most important factors for the success of a software development effort." [2]"XP also emphasizes open workspaces, a similar"people issue" that is outside CMM's scope."[3]3.2 OwnershipOnce Trust becomes the core of the working model, it results in giving more ownership to the team. The only expectation on the team would be delivering what they are good at, that is working software. "Scrum Team: Responsible for developing functionality. Teams are self-managing, self- organizing, and cross-functional, and they are responsible for figuring out how to turn Product Backlog into an increment of functionality within an iteration and managing their own work to do so. Team members are collectively responsible for the success of each iteration and of the project as a whole." [1]3.3 FulfillmentAgile prescribes delivering in short and frequent intervals."Our highest priority is to satisfy the customer through early and continuous delivery of valuable software" [15].There are multiple objectives that get met through this principle. It gives an opportunity to realize the benefits quickly, the trivial one being reduced time to market. It gives frequent retrospections for the team to reflect on their own progress and collectively alter their course for betterment, and gives an opportunity to demonstrate what is being built for the customer and get necessary feedback to align their course to meet customer's expectations.There is another aspect that is a direct result of these short cycles. The team gets to see the results of their hard work pretty quickly. This results in a team getting the sense of fulfillment at regular and short intervals. This makes a staggering contrast to ambiguous and abstract control charts under the supervision of a quality group that may not understand what the team is building.Success of a team in an Agile environment rests with team and not with Agile. The same cannot be said in a CMM atmosphere where the success is assumed and accredited to the process.4. Generation YA major portion of work force today belongs to a generation that is termed as Gen Y. These are the people who are born between 1977 and 2000. They are estimated to be more than 20% and are the fastest growing segment of the work force [19] . This is the generation that grew up with numerous TV channels, Internet, CDs, DVDs and cell phones. They are highly educated, tech savvy, know how to quickly get help and from anywhere. They like to own their work and control it. They are willing to accept the responsibility for what they own. They would rather take this responsibility than take orders from the top or work in a completely prescribed working model, for whichthey have no time .5. Why Agile suits better to Gen YGen Y has grown up educated, informed and formed opinions at a very early age.They have high expectations from their employers. They expect fair and direct managers [19]. They expect the managers to help them grow professionally. They cannot take a controlling manager who doesn't trust them.Gen Yers want to learn quickly, continuously and from their co-workers. They also want ownership and immediate responsibility. They want to control their fate and are willing to face consequences for their decisions. This makes it impossible for a CMM-like atmosphere to engage them where they get audited periodically for some compliance or other. In contrast, the same trait works perfectly to Agile as they can continuously get feedback for which they are directly responsible and have an opportunity in their own hands to take corrective actions.The best way for organizations to exploit this high energy generation is to provide such a relatively free and trusted environment. Organizations can set high level goals for Gen Y and demand quick returns. Organizations in fact can relieve themselves from trying to educate this generation on how to achieve the goals. This generation does not need those expensive heavy-weight models to make them work any more.This makes Agile a ready buy-in with Gen Y. The feeling that they are in control of what they are doing wins their hearts more than anything else.As Mary Christensen puts it "Corporations trapped in outdated ideologies are struggling to adapt to these free-spirited individuals."[17] Teams that are learning to trust individuals are adapting to a generation that's producing results struggle-free.


Main Ref: http://www.agilejournal.com/articles/17-articles/1736-a-critical-look-at-cmm-and-agile-through-gen-y

6. References
[1] Ana Sofia C. Marçal, Bruno Celso C. de Freitas, Felipe S. Furtado Soares, and Arnaldo D. Belchior1, "Mapping CMMI Project Management Process Areas to SCRUM Practices", http://www.cesar.org.br/files/file/SCRUMxCMMMI_IEEE-final03.pdf
[2] Srinivas Chillara and Pete Deemer, "Scrum and CMMI: A High level assessment of compatibility", http://www.etakey.net/articles/BnB-kleine-ScrumCMMI.pdf
[3] Mark C. Paulk, "Extreme Programming from a CMM Perspective", ftp://ftp.sei.cmu.edu/pub/documents/articles/pdf/xp-from-a-cmm-perspective.pdf
[4] Hillel Glazer, Jeff Dalton, David Anderson, Mike Konrad, and Sandy Shrum, "CMMI® or Agile: Why Not Embrace Both!", http://www.sei.cmu.edu/pub/documents/08.reports/08tn003.pdf[5] David J. Anderson, "Stretching Agile to fit CMMI Level3", Agile Conference, Denver July 2005, http://www.agilemanagement.net/AMPDFArchive/Agile_2005_Paper_DJA_v1_6.pdf[6] CMMI Product Team, "CMMI for Development, Version1.2", Software Engineeering Institue, CarnegieMellonhttp://www.sei.cmu.edu/publications/documents/06.reports/06tr008.html[7] Kevin Trethewey, "BLOG: The difference between Agile methods and CMMI", http://dotnet.org.za/kevint/archive/2005/11/21/48410.aspx[8] James Bach, "The Immaturity of CMM", http://www.core.org.cn/NR/rdonlyres/Aeronautics-and-Astronautics/16-355JFall-2005/A832C947-2CB2-47BE-9947-50993932EC28/0/bach_reading.pdf[9] Vasudeva Varma, Vamsi and Kirti Garg, "Managing Growth in a High Technology Venture - the Fact-Tree Way", http://www.canopusconsulting.com/resources/techarticles/facttreecasestudy.pdf[10] Nagaraju Pappu, "Response to the Fact-Tree Case Study- What is wrong with the Indian Software Industry?", http://www.canopusconsulting.com/resources/techarticles/whatiswrongwithindiansoftwareindustry.pdf[11] Ajay Batra, "What makes Indian Software Companies Tick? (CMM Practices in India)", QAI http://www.qaichina.com/Misc/thought_leadership/hkspin.pdf[12] QAI in the press http://72.14.235.132/search?q=cache:http://www.qaiindia.com/News_room/News/quality_bug.htm[13] Christine Davis, Margaret Glover, John Manzo, and Donald Opperthauser, "An Agile Approach to Achieving CMMI", http://www.agiletek.com/images/AgileTek/pdf/an_agile_approach_to_achieving_cmmi.pdf[14] "Agile Manifesto", http://agilemanifesto.org/[15] "Agile 12 Principles", http://agilemanifesto.org/principles.html[16] Gen Y on Wikipedia, http://en.wikipedia.org/wiki/Generation_Y[17] Mary Christensen, "Be a recruiting superstar" AMACOM publishing May 7, 2008[18] Carolyn A. Martin Ph.D, and Bruce "Tulgan Managing generation Y" HRD Press[19] Cara Spiro "Generation Y in the Workplace", http://www.dau.mil/pubs/dam/11_12_2006/11_12_2006_spi%20_nd06.pdf[20] Donald E. Knuth "Computer Programming as an art",1974 ACM Turing Award Lecture http://www.paulgraham.com/knuth.html