Controls
Controls help control the delivery of work.
Initially, it doesn’t really matter if we go over budget. We want to establish what the capability is to do work. To do that the team must estimate the expected effort to complete some work, then track the effort to complete the work. The team estimates work and the product owner assigns value to work. We want to know how much value the team can deliver by some future time. We start out not focusing on a budget control, but velocity and throughput control.
As we move along and get more mature, we assess how much was effort was spent and estimate how much is remaining on work in progress. With this knowledge we can asses the estimated budget against the actual budget against the remaining budget, including adjustments for remaining work. Now we are looking at budget controls.
At this point the team should be able to view the budget in terms of the value assigned to the work. The team is able to optimize effort expended to optimize value delivered. If the product owner is able to equate work with more than some liquid asset like project dollar cost the team can focus on more than meeting the original budget. Overtime the team will more consistently hit budget targets if they are allowed to not focus on budget.
Let's say we have something called a risk ration. When the time spent on a story reaches a certain percentage of the original estimate, the story is considered at risk. The original estimate is logged and everyday the time spent on the story is logged. Everyday, any time that was spent on a story is logged and that triggers a risk calculation. We sum all the time spent and subtract it from the original estimate to get the remaining budget. If the result plus the estimated remaining is more than the original estimate or if it is within a certain % of the original estimate, the story is considered at risk. Since it is at risk the risk is assessed, the story re-estimated, and the risk communicated to stakeholders. In this scenario the control is made actionable, automatable and capable of helping the team improve delivery.
How much business value and development complexity can we deliver per week, how much work can we deliver? Ultimately, this is what we want to know. As the team assesses effort the weekly effort will stabilize and we will be better positioned to create more realistic estimates.
When we deliver more value than we spend on delivering we profit. When we can establish a consistent amount of effort delivered we can estimate the value we can deliver. It will never be perfect, but not trying to measure and control it relegates the team to failure without learning from mistakes because they can see the mistake.
Observe behavior of production Validate predictions Hypothesize on the best ways to improve production
Story Data
Original Estimate (immutable)
Actual Effort
Remaining Effort
Stages
Start Date
End Date
Work Tracking
Board Work Center Stage
Any stage that requires human intervention to move to the next stage should be split to include doing and done stages. Example - when in the local stage when you are done you commit and it is automatically deployed to Dev so no need for doing and done because the work will never sit in the done stage. In Dev, when development validation is complete there is a manual approval to deploy to UAT so there should be a done stage to indicate what work is ready for UAT deployment. As the team matures and we trust automated checks more we can make Dev to UAT deployment automatic if checks pass and we can get rid of the doing and done stages.
Prioritization
Cards are stacked in order of priority with highest priority items on top.
Development Controls
Story Acceptance Check Automation/Regression Checks
Scope Monitoring/Feature Creep Prevention
Complexity Boundaries/Story, Estimate, Architecture, Code, Reviews
Prevent over-engineering
Refactor has to be covered by tests identified in dependency analysis on the change to be done for the refactor
Estimate Meeting
This meeting includes the product owner, project manager and the project team.
Estimate items in backlog
Backlog Meeting
This meeting includes the product owner, project manager, and any team leads that have tasks in scope for the meeting.
Review billable vs non-billable time for sprint
Review estimated vs actual vs remaining
Review throughput and lead time for sprint
Review quality
Review resource availability for next sprint
Prioritize Backlog
Review Meeting
This is a stand up meeting that includes the project manager and the project team.
Limit time of discussion per team member
Discussion of WIP
Review Card’s
Blockers
Time Taken
Time Remaining
Issues or Interesting
Metrics
Velocity
The project management system can tell us how many hours we logged to stories. We can surface a dashboard that shows how many hours the team logged to billable and non-billable work. I am not sure if we can identify billable vs non-billable without a team member selecting billable or non-billable as they enter time. It would be nice if we can just mark a task as billable on non-billable. Then we can automatically bill what's not over estimate and have a discussion to decide to bill or not bill time over estimate.
This control is not to beat the team up to increase billable hours. Just to inform the team if it is hitting its internal targets. How to improve is the only goal. Not who to blame. It is hard to calculate how to measure the value delivered to customers based on billable vs non-billable hours. How do you measure a 15 minute conversation that saves a project versus 80 hours invested in billable hours on a project that is running way over budget and late.
We only want to see how billable we are and how we are contributing to the overall company goals. Just because one person or team is more billable does not measure how important the team is. The entire company contributes to a customer wanting to continue doing business with us. The entire team contributes to the value that the company provides.
Throughput
We also want to know how many tasks are new, waiting, due, in process, and completed in the sprint. Actually, we could count the tickets in each column of he Kanban and report the count for each column and include how many of the tickets were added after the sprint started.
We want to see where we are bottle necked because we want to optimize the bottle neck. Optimize means improve the throughput of tasks through the bottle neck. That's were we focus process improvements. Right now we are focused on development process improvements, but there are work centers before and after development that may be or contribute to the bottle neck. So, we have to include them in our process improvement exercise. If we don't we could be just contributing to overloading the bottle neck with no real improvement in throughput across the entire value stream.
Knowing the count of tasks in various states allows us to identify the bottle neck and where to optimize our process. Again, the count is not a tool to shame the team into increasing the count of tickets done. We just want to know our throughput to help us optimize work centers and the entire delivery system as a whole.
We could count these in the project management system. We could track them on a Kanban in the project management system. The development task management system can be used to add link from the project management system to a source control system commit, but we rely on the project management system as the source of record for story state.
Once we can get counts and identify bottle necks its is time to get a real calculation of throughput. To calculate throughput we have to not only have the count of tickets in each state, but the date a story was started and the date story was completed in each state, a portion of the value stream and the entire value stream.
Count the days the story was active in the value stream. We only count the days for stories completed in the measurement period.
The throughput is the mean of the days spent in the system for each story completed in the measurement period.
Standard Deviation
We also calculate the standard deviation of the throughput so we can see how accurate the throughput is. The closer the size of the stories the smaller the deviation should be. Standard deviation lets us know how much we can depend on throughput for estimating. Higher deviation means less confidence and we should pad estimates based on throughput.
To calculate standard deviation we use population standard deviation because we are only concerned with the total population of stories actually delivered in a measurement period.
We start by getting the difference between the throughput days for each story vs the mean.
Next we get the square of each difference.
Next we get the sum of the squared differneces.
Next we calculate the variance by dividing the sum of the squared differences by the total number of stories delivered to get the mean or average of the squared differences.
Lastly, the standard deviation is the square root of the variance.
Team Profitability
One of our biggest issues as a team is being able to produce accurate estimates. Being able to identify where we go over estimate will help us to understand how we can improve our estimates. Keeping an eye on throughput, velocity, and how the team is doing on actual effort vs actual estimate can give signals to let the team know when to take a look at why the team is of estimated velocity, throughput and actual vs estimate.
Team profitability is a measure of the expected profit made on work delivered. To calculate team profitability you start with calculating the revenue for each story.
Calculate Sprint Revenue
We use estimates as the projected revenue. The estimates should be that same story estimates used to derive the project estimate in the project proposal and contract. We use the estimate because we are fixed bid and what we estimate is what we get paid.
Get the hour estimate for each story delivered.
To get the hour estimate for a story, get the hour estimate for each bill rate (e.g. Build Out, Backend, Testing...) used to estimate the story. Multiply the hour estimate for each bill rate by the bill rate to get the bill rate revenue for the story. Lastly, sum the bill rate revenues for the story to get the total revenue for the story.
You sum the total revenue for each story done in a sprint or measurement period to get the total revenue for the sprint.
Calculate Sprint Cost
Get the actual effort hours reported for each story delivered.
To get the actual effort hours for a story, get the actual effort for each bill rate used to deliver the story. Multiply the actual effort hours for each bill rate by the bill rate to get the bill rate cost for the story. Lastly, sum the bill rate costs for the story to get the total cost for the story.
Alternatively, you can look at cost as salary for each team member used to deliver a story. What is the cost of the team to deliver the story. To get the weekly cost of a team calculate the weekly cost of each team member. To do this take the yearly salary of the team member and divide it by 52 to get the weekly cost for the team member. Then get the total hours logged by the team member in the reporting week Next divide the weekly cost by the total hours logged to get the hourly cost for the team member. Lastly multiply the hourly cost by the actual effort hours logged by the team member on the story to get the total cost for the team member to deliver the story. Then sum the total cost for each team member to get the total cost of the story.
The alternative method to calculate the sprint cost is closer to the actual financial implications of salaries as opposed to using bill rate, but it is harder to calculate and requires visability into team member salaries.
After you get the total cost of the story, you sum the total cost for each story done in a sprint or measurement period to get the total cost for the sprint.
There are other costs like software and hardware costs, training, travel and the like, but the focus is team profitability.
Profitability
Subtract the sprint cost from the sprint revenue to get the team’s profitability index. Watch the index and make adjustments to keep this number stable or rising, falling is an indication of an issue.
Quality
The count of issues reported in production and UAT.
Lead Time
Lead time is the time it takes work to flow through the system.
To provide accuracy in lead time calculation it should be calculated on similar types of projects and work within the projects.
Types of Work
New
Change
Bug
Cycle Time
Amount of time to complete a single story. Unlike throughput which doesn't account for work in the backlog this is a measure of the time between the date the story enters the backlog and the date the story is complete. So this is more than the hours actually worked on a task. This is also applied to each story categories. The difference between a story entering and exiting a category and the time between a story exiting a category and entering the next category. This all aides in monitoring the throughput of the system and identifying bottlenecks that need to be optimized. There will always be a bottleneck. When you optimize on bottleneck the bottleneck will move.
We want to know how much value can be delivered or how much work can be done per time period by the team. We also want to know how efficiently we are addressing customer needs. Waiting in the backlog is like waiting in line, there isn't much value in it and if you wait to long you get frustrated.
Reporting
Communication
Team
Product Owner
Project Manager
Creative Manager
Development Manager
Customer Acceptance Manager
Delivery Manager
Lead Developer
Design Developer
Front End Developer
Mid/Senior Developer
Automation Developer
Say we have these active team members:
Project Manager
Lead Developer
Developer
The our total lines of communication is
Add another team member, say Customer Acceptance Manager and the total lines of communications doubles
Every time we add a team member the lines of communication increases and it becomes more difficult communicate through the delivery process.
Culture
Honesty
If someone is going of process, correct them. Nothing personal, we got shit to do. Not saying that we can’t have a little fun, but no deep technical discussions… take that shit off line.
Last updated