What is the Product Backlog and How to Optimize it?

A Project To-Do List

Every summer, when a close friend of mine has got a couple of weeks off work, he loves to take on a brand new DIY project at home. This year it’s something quite different again; re-plastering the crumbling front wall of his house. Each time, with whatever job he embarks on, he makes sure to first compile a complete list of the things that need to be done within the project. Although he doesn’t enjoy the benefits of having a team of people to help him, that to-do list remains his most important tool.   

Similarly in Agile, the one tool that stands out as being more basic and more important than all the rest, has to be the Product Backlog, which in essence is the methodology’s to-do list or as some prefer to think of it, and perhaps they are more accurate, a wish list. In the culture’s terminology it is described as “an ordered list of user-centric requirements that a team maintains for a product.” User-centric requirements being a full list of the requirements needed for a system, project or product. A comprehensive understanding of the product backlog and its correct utilization, whilst realizing its most likely potential problems, are the keys to unlocking your project and setting it on the road to success. And here remember, you do have a team of people to help you.  

The Product Backlog

Contents

As with any list of things to be done, the product backlog could be described as the single central source of things that are to be worked on and is the key connection between the Product Owner and the development team. Derived from the Product Roadmap and its requirements, it is a list of new features, any changes to existing features, infrastructure changes, bug fixes, non-functional requirements, or any other relevant activities thought needed to get the desired result. Everything on it should be attempted by the team, even if there is no guarantee that they will all be delivered, whereas anything that’s not on it, should not be undertaken. Think of it as a list of possible options broken down into a series of steps in a project rather than full commitments.

However, illustrating perfectly why agile is called agile, the product backlog is adjustable to fit the work of the team. Although it should theoretically be reducing in size as tasks are completed, removed, and the project develops, it does allow the opportunity for new items to be added and to ensure that prioritization is correct. These regular reviews of the backlog, known as grooming, make up part of its optimization.    

Format

In terms of its layout, the product backlog prioritizes and orders items, sorted by their business value and risk, by showing the most important ones at the top then descending in steps to those of lesser significance or of a less refined nature. The high priority items towards the top being referred to by the self-explanatory description “fine-grained” and those towards the bottom “coarse-grained.” Therefore, the team is aware of what to deliver first, the items at the top of the list, although sometimes lower-priority items are picked off early where time allows. Think again about the to-do list if you were plastering your front wall at home; you may have time to get a less relevant job done whilst the mixed plaster in your bucket is settling to the correct consistency.

In order to compile the product backlog, the most widely used format is via the utilization of User Stories although other methods such as User Cases are available. Story points are used for each item since their degrees of complexity vary. The Definition of Done also has a strong influence and, through calculations of velocity in relation to the product backlog, a release plan can be formulated.

In physical terms, the product backlog can be depicted on a physical board or, more often nowadays via a spreadsheet, which also includes a column highlighting in which sprint each item will be addressed.     

Managing the Product Backlog

The Product Owner

As the key stakeholder, and the person responsible for maximizing the value of the product and ultimately accountable for the end product, the product owner has a complete view of the project and is therefore, most appropriately, the main creator of the product backlog. He is the direct connection with the customer and the needs of the customer and, as such, plays the major role. It is also his responsibility to convey its contents clearly and thoroughly, with no ambiguities, in order to avoid any misunderstandings amongst the team. Similarly, he must prioritize the tasks, namely he is the person who puts them in order of importance, based on added value, costs and risks, and his consent is needed should any changes be required through the course of the project.     

The Scrum Master and the Team

Although the Scrum Master has many responsibilities, his involvement in the product backlog is less intense. This is so because, although he may appear to be the ideal person to head the prioritization process, his role is to assist in the process not the product itself. However, as the project develops the scrum master and the product owner should work together in order to ensure that when a task moves from the product backlog to the sprint backlog the team fully understands what is required to fulfill that task.    

The role of the team where the product backlog is concerned is to provide a certain degree of input as regards prioritization but more importantly to select items which they are able to complete in the forthcoming sprint. Working with the product backlog means not being restricted solely to its content but having access to and using the range of other tools at the disposal of the team and scrum master, for example, workflow descriptions, interface guidelines and task boards in order to add more detail. Project management tools in scrum are designed to be complimentary to one another.

Optimization of the Product Backlog

Once the contents and format of the product backlog are understood in addition to the roles of the people involved, how can it best be optimized?

Structuring the Product Backlog

When you write down ideas for any project you are intending to do or working on, it is a good idea to group certain items together into more general but related themes. The same applies to coarse-grained items within the backlog, where access to information is made easier, prioritization benefits and a more understandable structure is created. Each of these themes could include up to five or six items therefore providing more general information as to what is needed to create the product.

Simplicity and Flexibility

In agile, and scrum in particular, the concept demands simplicity and flexibility wherever possible. Compiling the product backlog does not require considering every single detail from day one since, as the project rolls on, more ideas will develop through customer, user and team feedback. Focusing on what is more critical and the positioning of fine-grained and coarse-grained items within the backlog illustrates this.

As regards flexibility, the 12^th principle behind the Agile manifesto actually says; “At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.” With the product backlog the same should naturally apply and the product manager needs to forget old-fashioned methods of project management in which, quite probably, few if any changes were made through the course of a project. As those involved discover more about customer needs and being able to satisfy them, original requirements should be altered or eliminated altogether. Regard the items on the product backlog as your to-do list but remember that they are not set in stone.        

Good Grooming

Mentioned earlier, Backlog grooming and good backlog grooming plays a vital role in the optimization of the product backlog. Just like in any project, think again about plastering a wall, this has to be an ongoing process. As items are considered done, they are removed and the new ones of most importance are moved to the top to be ready for implementation in the upcoming piece of work, ie the next sprint. New items are added, existing ones changed and any incorrect estimates are altered.

The Grooms

In order to be more effective, the grooming process can be undertaken by the product owner, scrum master and, in fact, the whole team as a collaborative effort. Although as stated before, we tend to view the product backlog as more of the domain of the product owner rather than others, joint participation in grooming may be far more productive. Face-to-face communication in this process is advised and by creating the conditions for open dialogue, there will be an increase in clarity regarding requirements and the obvious opportunity for an exchange of knowledge. This grooming process does not have to take place in a separate fixed meeting, it can be held briefly after the Daily Scrum or thought of as a prerequisite of a good sprint planning meeting.

Potential Problems with the Product Backlog

In order to optimize the product backlog, one must not fall foul to the potential problems which exist. Here, misunderstanding of the tool and its usage because of either the terminology used or the responsibilities of those involved are arguably the two most likely dangers.   

Different “Backlogs”

In agile methodology, a complete comprehension of terminology and application of techniques are of the upmost importance and the main danger could be in the use and misinterpretation of the word “backlog”.

The product backlog has been explained, but it can easily be confused with the Sprint Backlog. The latter, although related to the former, is a different tool. As a subset of the product backlog, the sprint backlog contains only the item or items to be completed during each sprint. At the planning meeting, items are discussed and moved from the product backlog to the sprint backlog. Therefore, the team is only dedicated to those tasks for that sprint and they remain unchanged for the course of it unless altered as a result of the sprint planning meeting. Should any items be unfinished by the end of the sprint, they are added back to the product backlog for the next sprint.

There are even more “backlogs” about. The Scaled Agile Framework (SAFe) backlogs, whilst trying to replace and improve the traditional product backlog, may create even more confusion so, despite being advantageous for established and experienced organizations, are probably best avoided by new agile teams.

Misunderstanding the Roles

Allocating specific roles to all the characters taking part in agile processes, and in particular scrum, is a fundamental. If the product owner does not adjust the product backlog as feedback from developers and stakeholders appears, the process loses its agility. Not allowing the scrum master or team members full access to the document is another danger. If it is not shared, interested parties cannot get updates on a regular basis. In addition, whilst the development team should be the one dictating velocity through the backlog, the product owner may well try to over-accelerate the work in order to match his initial prioritization.  

So, if you can get a full understanding of what makes up the product backlog, your to-do or wish list, then use it effectively and avoid any potential problems, you should be set to achieve a near perfect project. Hopefully, by early September, my friend’s re-plastered front wall will be equally as smooth and solid.