Agile Practices


While it is always important to remember that Agile is a set of values and principles, it can be useful to look at some of the practices that teams implement when following Agile. (If you haven’t read What is Agile? stop and read that before this article.) Practices are only Agile to the extent that they help a team follow the values and principles. In this post we are going to look at how a mature Agile team does their work. There are a wide variety of practices used by Agile teams. We are going to focus on the general aspects of some common practices.

Your team may arrive at something similar to support Agile principles or you may end up with practices that look very different. Just don’t forget that the important part is the why behind each practice.

Agile Team

Agile software development relies on a strong team. In this section we are going to look at some aspects of creating and nurturing a team that supports following the Agile Principles.

Team Composition

Agile principles suggest that the best way to write software is when you have face-to-face communication every day. In many organizations, the best way to do this is to make a team of everyone that is needed and put them into a big room to work together every day. So a team would consist of developers, business owners, and testers. Sometimes the business owner role has to be filled by a representative, but the idea is that you have people working together every day that wear all the various hats that the project needs to make progress.

Supporting Agile

It is important for the individuals on a team to buy into a different way of working. Some places have had success by forming a new Agile team by asking select individuals if they want to join. However, a new member has to sign an agreement that includes the Agile Principles as well as some other language about being willing to participate in a team driven environment.

When you try to fit Agile onto an existing team, it is easy to run into resistance from people who aren’t interested in taking a new approach to software development. It is not uncommon for a few people to wreck the progress of a team that is trying to follow Agile Principles by being unwilling to participate in a team-driven culture. While most people find the switch to Agile challenging but very rewarding, one detractor can bring progress to a halt if left unchecked. The easiest way to avoid this is to do your best to make sure there is support up-front for the way that the team will be working and making decisions.

Cross Trained Teams

Valuing face-to-face interactions lends itself to cross-functional team composition. Good communication leads cross-functional teams to do a great deal of cross-training—even if it isn’t formal. If a team is truly focused on delivering valuable software rapidly to the customer, they can’t have silos of who can do what. That doesn’t mean that the business owner on the team is going to learn to write code (though that does happens sometimes) but it does mean that working together on the same team should recreate organizational silos in a smaller setting.

Practices like pair-programming can help promote this level of collaboration and cross-training. In a well functioning team, individuals will be constantly sharing how to do new things. It is a good sign when you hear people asking to help with things they haven’t done before so they can learn.

Defining Work

The way a team defines their work can heavily influence how they approach doing the work. If the work is defined in a way that supports Agile Principles, it is much easier for the entire project to follow on that foundation. There are many ways to do this, but here are some very general approaches that a number high performance Agile teams have found work well in their organizations.

Story Board

Most teams find that in order to follow the principles regarding communication, they need some type of way to visualize work. This is often done with some type of story board where work can be represented on cards by placing them in different columns to represent the state of that work. The principles tend to suggest a few different states (columns) that are good to track visually.

First you need some state that shows work that is yet to be done. Typically this is called the backlog. Since we have a principle that we are going to focus on delivering valuable software, this backlog is usually prioritized with the customer deciding what the most valuable thing to put at the top of the list. If the software can be kept mostly in a state where it can be released into production, then the code that has been developed can be put to use whenever the customer wants to start using some of the value that has been created. The prioritized backlog helps make sure that the next item the team works on represents the most valuable thing to the customer.

After the backlog, there is usually a state of work that says it needs to be discussed with the customer. This is where the developers, customer and testers get face-to-face and walk through what it means for the particular piece of work to be done. This state can be called requirements, discuss with user, or whatever makes sense but it should involve face-to-face interaction between the people who will use the code and the people who will create it.

The next state is usually doing the actual work. So a name like in progress is appropriate. This is where the work should go that is actively being developed. Once the work is done, there should be a state where it gets shown to the user to make sure that what was built was what they actually need. This state can be called verification or something similar. Once the user agrees that the work meets their needs it gets moved over to the done column.

This is a very general approach and some teams find it works well to add rows (swim lanes) to represent different types of work or additional columns to handle other states that are necessary in their organization. Others add columns that represent when the work gets deployed to staging and to production. The way work is represented should be malleable and constantly adapting to the needs of the team.

While we’ve described a physical board, there are a number of tools to create virtual software boards that allow this approach to visualizing work and can be especially helpful with a distributed team.


Most Agile teams define their work as stories of some sort. The reason for calling them stories is because it helps promote the value of concentrating on user collaboration rather than contracts. A “requirement” has quite a bit of baggage associated with the term that tends to prevent collaboration. A story needs to be something that can be written quickly as a placeholder for work with the idea that it can be refined through collaboration when it is time to actually do the work later. Many teams find that using a simple template can help keep the stories short and easy to create. Here are two examples:

As a visitor
I want to create an account
So I can apply for a loan

As a registered user
I want to view my order details
So I can I can see when my items will arrive

There isn’t anything special about the template other than making sure it captures who, what and why. This is just a way to help enable communication and get people on the same page for the collaborative discussions that should happen before beginning the development work.

Developers tend to write stories from the developer perspective. So you might see a developer write a story about some section of programming they want to have built. For example:

As a developer
I want to have a database
So I can store information

Obviously the application is going to need to have a place to store data, but the problem with this story is that it doesn’t keep the work focused on what the user considers valuable. We are following a principle of delivering valuable software to the user, so we need to organize our work in keeping with this value. One of the easiest ways to do this is to make sure that the stories represent how things will be different in the user’s world when they are completed.

Developers tend to see applications as different layers. There might be a database layer that is underneath a business rules layer and that is underneath a GUI layer. Users, on the other hand, see an application as representing something they do. They see it as a piece of functionality that cuts through all those layers.

So developers see the app as the layers of a cake with each layer built on the one below it. Users see an application as a bunch of slices of cake with each slice letting them do something useful. While it may take some effort for developers to get used to building an application the way users view it, it is very possible and the ability to align the development of the work with the user’s definition of value is a powerful way we can follow our principles.

User stories should be written as slices that represent complete functionality that a user can look at and use. For example, instead of building a database layer, we can start with a story that speaks to the ability to login to the system—building just the parts in the different layers necessary to support that slice of functionality.

Doing the Work

As an Agile team evolves, they will develop, adopt, or borrow a number of software development practices that can be used to help them follow the Agile Principles. In this section we will look at a few development practices that many teams find helpful.

Pair Programming

Pair programming is a common practice used by Agile teams. It puts two people at one computer working on writing code. While it might seem like everything will take twice as long to code, programming is a thinking activity more than it is a typing activity. Two developers are less likely to get stuck and more likely to catch problems as they occur. Many teams have found that for some types of development, pair programming is actually faster than the alternative.

Another advantage of pair programming is that as people rotate through different pairs, skills are transferred through the whole team. If one person finds a keyboard shortcut that saves them time, the person they are pair programming with will pick it up and show it to the next person they program with. Pair programming is an effective way to continually increase the skill of everyone on the team.

New teams that are not used to working collaboratively can especially benefit from pair programming—especially when they are first formed. If everyone is required to always write their code as a pair, the team develops good communication skills and much lower resistance to asking someone else for help. Often after a few months of required pair programming the requirement can be lifted and the team will naturally continue with a much higher degree of interaction than they had before. Even if they aren’t constantly programming in pairs, they are much more prone to ask someone to come over and collaborate after having pair programmed for a time.

Continuous Integration

Since Agile says that our highest priority needs to be continuously delivering valuable software, anything we can do to streamline the delivery process become very important. Automation helps follow this principle. Continuous integration is the process of automatically building and testing the software every time a developer checks in code. This helps reduce the amount of time from when a bug gets checked into the code until it is noticed. A common problem in software is for a developer to check in a bug that works on their machine, but fails for everyone else. A continuous integration server will run the changed code and notify the team of the failure so it can be fixed quickly, while the developer still has the context for what they were working on when they made the change.

The longer that elapses between a bug entering the code base and identifying it, the more time it will take to fix. Continuous integration helps keep this cycle as short as possible.

Test Driven Development

Several of the Agile principles deal with creating good design. Others talk about the need for technical excellence and the ability to continuously deliver new functionality. Test Driven Development is a practice that most teams find support all of those things. TDD involves writing tests that would pass if the code you’ve imagined worked correctly and then writing the imagined code until it exists in a state that will make the tests pass. Not only does this approach help maintain focus on the goal of the current work, but it also provides a great deal of safety when code needs to be changed. When a developer can make sweeping changes through the application and at the press of a button verify that no unintended side effects have been created, it is much easier to respond rapidly to changing requirements.

TDD also can help developers keep things simple and not over-engineer their code. They can choose the simplest approach and get the software doing what is required functionally with tests to prove it. If they then discover something needs optimized or requires a more complex approach than what they tried originally, tests make it easy to tell if the more complex approach still functions the same as it did when it was simple.

Behavior Driven Development

While Test Driven Development focuses on making sure that the pieces of code do what the developer expects, Behavior Driven Development focuses on making sure that the application as a whole does what the user needs. BDD is a practice that helps teams follow the Agile Principles that discuss creating good communication with users.

There are different tools to facilitate BDD, but the key piece is creating a plain English description of what the software needs to do that users, testers, and developers can all agree on and then making that description into something that can be executed against the application. This means you have a specification (the description of what the software does) that everyone can understand, but it always stays current because if it stops describing the application’s behavior, the build breaks.

The key point of BDD is that it enables better communication with the users and provides a way of clearly articulating and specifying what the application is supposed to do along with a mechanism to automatically verify that it behaves as expected.


Most teams will naturally develop some rituals in the way they work. In this section we will look at some of the rituals that many Agile teams have found useful as a way to follow various Agile Principles. The value isn’t in the rituals themselves, but in the principles they follow. A highly functioning Agile team will probably tweak and modify these rituals when they find changes that will let them better follow Agile Principles in their particular environment. These rituals are a good place to start as long as the team understands the principles behind them.

Daily Standup

To follow the Agile principle that says the most efficient way of communicating with and within the team is through face-to-fact meetings, many teams find it useful to establish a ritual that helps facilitate this communication called a daily stand-up. The idea is to get everyone together for 15 minutes and walk through the current state of the work.

One approach is to go around the room and ask everyone to say what they worked on yesterday and what they are working on today. Many teams follow this approach, but running a daily stand-up that is organized around the individuals usually results in meetings that aren’t driven by user needs. Individual centered stand-up meetings tend to spend more time discussing issues from the greater organization than on topics that help move the work forward. The user needs should be represented by the user stories. Stand-ups that focus on the user stories keep the actions of the teams better aligned with the Agile Principles and keep work and discussions focused on the value being provided to the user.

Once the team has heard a brief update about each user story, it is usually a good time to make any general announcements to help coordinate the work. For example, a team member may find it useful to remind everyone that they will be out on vacation for the next two days, the team lead may want to remind everyone to fill out their time sheets, etc. Once the team has covered the stories and any announcements, there will be people who need to get more details or talk through some issues. When the stand-up is over, these discussions can take place with the more detailed conversations they need to move forward on their work.

Stand-ups typically should last about 15 minutes and many teams conduct them standing up because it is easier to keep the meetings short if everyone is standing. Standing up also helps facilitate the interactions after the meeting where team members need to continue a conversation about a particular story.

The stand-up should keep everyone up to date, but you don’t need to have a 20-minute discussion about each story—especially if it isn’t relevant to everyone in the room. A few questions and clarifications are fine, but if it looks like the discussion of a particular story is taking too much time, it is usually best to move on and let the people involved dig deeper after the stand-up is over.


The Agile Principles say that teams should regularly reflect on how to become more effective and adjust behavior accordingly. This principle is usually followed through the use of retrospective meetings. A retrospective is a meeting where team members discuss what is going well and what can be improved. Some teams do these to coincide with the rhythms of other rituals in the project. For example, if a team is batching their work into two week periods they might do a retrospective after each period. Teams that approach their work as a continuous flow, may still choose to do regular retrospective meetings every two weeks or some other interval. Some teams find it effective to just let team members call a retrospective whenever they feel one would be helpful.

There are many different ways retrospectives can be conducted. One simple approach is to give everyone two sticky notes and ask them to write one thing that they feel is going well and one thing that they feel needs improvement. Group the notes on the wall in these two categories and then let people vote for which “needs improvement” notes they want to address. Take the top voted items and spend some time discussing some concrete steps that will improve things.

These changes should usually be things that are small, manageable and something that can be rolled back if it doesn’t seem to help. The goal isn’t to fix everything—just to make adjustments to areas that need improved and keep the parts that make things better.

User Demonstrations

Several of the Agile Principles talk about focusing on working software that the business owner can actually use. Having regularly scheduled times to demonstrate software for stakeholders and users can help teams follow these principles.

These demos can be formal demonstrations in a conference room, every two weeks or something much less formal that just involves getting the users around a developer’s computer and letting them use some new functionality.

The “right” way of doing it depends on your team and your users. However, most teams find they get the most out of demonstrations when a user is actually the one “driving” the application. If the team has users embedded in the team, those users are an excellent choice to demonstrate the application to other stakeholders.

What is Agile?


Many things get called Agile—especially by people who are selling something. Agile is defined in the Agile Manifesto. The Manifesto makes it clear that Agile isn’t a methodology. It isn’t a specific way of doing software development. It isn’t a framework or a process. In fact, most of the things that are marketed as Agile tend to miss the point of what Agile actually is.

Agile is a set of values and principles.

Much of the discussion around Agile has to do with following different practices, using various methodologies, and even developing with specific tools. For example, while a team may find that having a daily standup is helpful, the standup is only “Agile” to the extent that it is the result of a team following the Agile principles and values.

When you understand this, it is easy to see that Agile is really a collection of beliefs that teams can use for making decisions about how to do the work of developing software. While this means the term Agile gets subjected to a great deal of abuse when people claim that this or that is the way to be Agile, it also means that if you truly understand what Agile is, it is surprisingly flexible. Agile doesn’t make decisions for you. Instead it gives a foundation for teams to make decisions that result in better software development.

Agile Manifesto

The Agile Manifesto is only 68 words and very simply says that the signers have found that they can develop software better by valuing the items on the left side of the list more than the items on the right side. Below is the Agile Manifesto:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Agile Principles

In addition to the values of the Manifesto there are 12 principles that support the values. Once again the principles are very general and are less about telling you what to do than they are about giving you the ability to make a good decision in a particular situation.

The principles are:

  1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
  2. Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
  3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
  4. Business people and developers must work together daily throughout the project.
  5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
  6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
  7. Working software is the primary measure of progress.
  8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
  9. Continuous attention to technical excellence and good design enhances agility.
  10. Simplicity–the art of maximizing the amount of work not done–is essential.
  11. The best architectures, requirements, and designs emerge from self-organizing teams.
  12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Agile Decisions

Since Agile is a collection of values and principles, its real worth is in giving people a common foundation for making decisions about the best way to develop software. For example, consider a new project that is in discussion on how to get the requirements from the business owner. The suggested approach is to require that the business owner write down all the requirements and sign off on them before beginning the work. A team that is following Agile would say:

“While that might work, isn’t that inconsistent with our belief that we should value customer collaboration over contract negotiation? And doesn’t it violate our principle that says the developers should be working with the business owners every day? How can we make this decision in a way that is consistent with our values and the principles we follow?”

Or consider a developer who is working on implementing a feature for the business owner. The developer realizes he needs a database to make the feature work. The first idea that comes to mind is to stop work on the feature and build out a robust database layer that will handle the needs of the feature and provide support for other development that will be needed later. If the developer believes in the Agile values and is trying to follow Agile principles they would think:

“But building out this layer means I will have to delay delivering what the customer can see as valuable software they can use. If I can find a way to build just what is necessary to deliver this feature, it will better align with my principles.”

When you have a team that is following Agile they will be making hundreds of decisions each week in the way described above. That is what it means to be Agile. Making each decision based on the principles and values that the team has decided to follow.

Being Agile

The decision making process matters. You can’t try to short-circuit things by taking decisions made by another team and just blindly doing what they decided to do. Another team may make decisions based on the Agile principles and values and end up with a particular way of doing their work. Simply trying to mimic another team’s actions and practices won’t make your team Agile.

After World War II Melanesian islanders were observed trying to bring cargo planes and their supplies from the sky by mimicking the practices they had seen performed during the war. This included clearing the forest to make a landing strip complete with full size planes made out of straw. They also created structures that mimicked a control tower out of bamboo and had someone sit in it wearing headphones fashioned from coconuts.

It is easy to fall into a similar type of cargo cult mentality when it comes to Agile. The things that are easy to notice in a highly functional Agile team are the practices they are using. But the practices a team uses is the result of following Agile principles and values. It is less important what practice a team happens to be using than why they are using it. In fact, as time goes by, a good Agile team is probably going to change and refine what practices they use.

A team might start with SCRUM and later find that Kanban is a better fit for delivering value to their customers. A team might begin standing up in a daily meeting and later decide it works better for everyone to stay sitting down. Another team might start out using Planning Poker to estimate story size and later do away with story points and simply split stories to where they are somewhere in the same range.

That isn’t to say it is useless to look at practices being used by teams that are performing well, but you can’t go looking for practices to make you Agile. Your principles and values are what will make you Agile. You have to look for practices that support your principles and values. The way you select your practices is what determines whether you are being Agile or not. If a practice is being selected because it looks like a good way to follow Agile principles, it is probably a good place to start. The same practice can work poorly for a team if it is selected for the wrong reason.


So what is Agile?

Agile is a set of values and principles.

How does a team become Agile?

They make their decisions based on Agile values and principles.

The decision making process is how a team becomes Agile. The values and principles have enough flexibility to allow teams in a wide variety of organizations to develop software in the ways that work best for their particular situation while providing enough direction to help a team continually move toward their full potential.

Test Driven Development


When you start using Test Driven Development, it is easy to get so excited about having regression tests that you miss the point of using tests to drive the actual development process. While regression tests are a nice benefit from following TDD, they really aren’t the main point. TDD is about letting the design of your code be driven by the following cycle:

  1. Write a simple test that will fail.
  2. Write simple code to make the test pass.
  3. Refactor to improve the existing functionality.
  4. Retest to make sure functionality is still the same.
  5. Back to step 1.

The goal is to follow a discipline that will produce code that is simpler and more robust than what would would emerge if you were to focus on a larger part of the problem.

You cannot use TDD successfully if you believe that a line of code should only be touched once. TDD requires you to do a lot of writing and rewriting. You may write code that makes your first test pass knowing that it will be rewritten when you add the second test. At first this can be frustrating. Why would you write code that works for test A knowing it won’t work for test B that you are going to write 90 seconds later? Why not go ahead and do it right the first time?

The reason is simple. You want to minimize the amount you have to think about at any given time and also minimize the amount of time that your code is in a broken state. The more things you try to do before getting your code back to where it runs, the longer the code exists in a state where it won’t run correctly. If you add a method that simply returns a hard coded value, it might seem like you’ve accomplished nothing in that step of getting the test to pass. However, you’ve made several decisions about the new method: Where should I put this method? What should I name it?, etc. When you add another test that forces your method to actually do something intelligent, you’ll still be working with a smaller set of changes than trying to do it all at once.

Once a test is passing, that doesn’t mean you are ready for the next tests. It means you are ready to refactor. You are now in a working state so you can change things and immediately tell if it changed the way the application works. Once a test passes you need to consider if there are any refactoring steps you can take to clean up what you have so far. Have you named things in a way that will be clear to others? Can you break some of your code out into a helper method to make it easier to understand?

This is probably the most important part of TDD—taking an application that is passing the existing tests and improving the code while leaving the functionality unchanged. If you aren’t trying to do this before each new test you write, you aren’t doing TDD.

Agile in Name Only


Last minute travel plans had me looking for a way to get some dry cleaning done with faster turn around than what my local dry cleaner could provide. We had to go to Joplin for a few hours before heading to Kansas City to catch our flight. A quick search showed there was a “1 Hour Dry Cleaners” in Joplin, so I figured I could drop my clothes off with them and pick them up as we headed out of town. On our way I gave them a call. The conversation went something like this:

1HDC: Hello, This i 1 Hour Dry Cleaners.
Mark: I’ve got some clothes I need to get cleaned quickly. Which location should I take them to?
1HDC: It doesn’t matter.
Mark: Ok so if I drop them off around 8 I can pick them up a bit after 9.
1HDC: They probably won’t be done. But I guess we could mark them as “expedite.”
Mark: Ok I just need them before we leave town, so if I drop them off at 8 and you mark them as expedite, will they be done by noon?
1HDC: Well even with “expedite” we probably won’t have them done that fast.
Mark: I’m sorry who did I call?
1HDC: This is 1 Hour Dry Cleaners in Joplin.
Mark: … ok…
1HDC: Oh, but that is just our name. It isn’t what we do.

I see a lot of “Agile teams” that suffer from the same problem. You aren’t Agile because you call yourself that name. Ceremonies don’t make you Agile either. A team isn’t Agile just because you have a daily standup or retrospectives. You have an Agile team if you are able to deliver usable code on a regular basis and welcome changing requirements. You have an Agile team if you prioritize face to face communication and the team has a lot of latitude to self organize.

Teams that are Agile in name only are just like the 1 Hour Dry Cleaner in Joplin that doesn’t do dry cleaning in an hour. No matter how great your name, it is the execution that matters.

(photo credit)

Teaching Class on Git


I spent some time this past week teaching a class on Git for the Treasury Department. We had a lot of fun. I used Digital Ocean to spin up an instance for the students instead of the AWS instances I usually use. It seemed to work well and in some ways it was simpler than using AWS.

Mark Shead Teaching Git Class

Mark Shead Teaching Class on Git Version Control

Why Do Some People Hate Agile?


Why do people hate agile?

Why do people hate agile?

Imagine that there is some group of people that speak English, but never learned to read or write. Someone literate shows up and explains that you can make marks on paper to represent words and read them back. All of the illiterate people are very excited about reading and writing and decide they need to start doing this right away.

Some of the people start learning phonetics and practicing writing the alphabet in order to learn the principles necessary to be literate. It is slow going and very difficult work.

Another group of people start making random marks on paper to represent the words they are thinking in their heads. They stand up in front of each other and hold a book open while pretending to read the pages. Of course they can’t read so they just make up stories. They try their best to emulate all the practices they saw people doing that could read and write.

After a year how do you think the two groups describe literacy? The group that had focused on the principles behind reading and writing would be very happy about it. They can pick up a book and read it. They can communicate with each other by writing on a piece of paper. The group that tried to emulate the practices without first learning the principles would declare literacy a fraud. Both groups would be equally strong in their opinions.

Agile is a set of values and principles. Teams use these to make decisions that lead to good project outcomes. These values and principles guide the selection of different practices.

There are a lot of people trying to “do Agile” by emulating the practices they have seen on other teams. They try to use Scrum, Kanban, Extreme Programming, etc. But these practices aren’t Agile unless they are being driven by the Agile values and principles. Like the people who try to develop literacy by pretending to read and write, they get very frustrated.

Agile is polarizing because it is easy for people to spend a lot of effort doing things that look like Agile without having any of the foundational beliefs in place. If someone says Agile is a recipe for failure, ask them to describe Agile. If they name a bunch of practices and no values or principles then it isn’t actually Agile that they dislike.

Mark Shead’s Home Office


For the past 13 years my primary office has been in my home. During this time I’ve experimented with a lot of different things from simple small desks in Mexico to large glass topped workspaces.  Here is what my current desk looks like.

Mark Shead Desk

The first thing people usually notice is the weird lights. I work on video conference pretty much 8 hours a day right now. The lights help make sure that the video conferencing is as effective as possible. The studio lights were about $150 and give nice even lighting for the video as well as for the work surface. Each stand has two 80 watt CFL bulbs which are supposed to be equivalent to about 300W in a traditional light.

The monitor on the right is hooked to a small Windows 8 computer that is running the video conference software. You’ll have to click on the image to see it better, but there is a small Cisco video camera on the monitor.

The computer on the left hand side is running linux and the laptop hooked to the center screen is my MacBook Air. If you notice there is only a single keyboard. I use a program called Synergy to share between the screens. My MacBook Air is the server and lets me move the mouse and keyboard to the screens on either side. This may sound like a small thing, but it is a huge improvement to be able to treat multiple computers as one from your input device.

The desk is a Turnstone Bivi. My favorite thing about the desk is that it has lots of places to hide wires. The top half of the work surface is a tray that I use to put my power supplies and cables and it gets many of the wires out of the way. Also the desk has 16 outlets built into it–and I’m using all of them.  I have the Bivi double so their are actually two work surfaces facing each other. Last summer when I  had an assistant helping me she used the opposite desk, but now I use it for my printer or if I need some additional room to spread things out.

Mark Shead in Home Office

This picture is looking from my desk backwards. The desk to the side is my old glass top desk and there is another one on the other side as well. I’ll sometimes use these for organizing paper work but often they start collecting junk so I may replace them with some shelves or storage units.

Agile Principles vs. Agile Practices


When people see a successful Agile team, the first thing they start looking at is all the practices the team is doing. They notice the daily stand ups, iteration reviews, test driven development, continuous integration, etc. Then they try to go replicate the success by copying all the practices they see being done. The problem with this is that for the practices to really be effective, they need to grow out of the Agile principles.

Continuous delivery of software by itself is of less value than when the practice grows naturally out of trying to follow this principle:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

Enforcing a 40 hour work week is a whole lot different than trying to follow this principle:

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

You can’t just blindly emulate a few practices that are working for others. Without understanding the principles of Agile, you will miss the fountain necessary to adapt different practices to suit the special things that make your team unique. Most of the benefit of Agile comes from its ability to help teams work effectively and efficiently regardless of all the differences.

Tapestry 5.2.4 Out


I have been doing quite a bit of work with Apache Tapestry.  In the past week, Tapestry 5.2.4 has been released and is worth checking out. Coming from a WebObjects background, Tapestry seems like a very logical approach to web development.  If you aren’t familiar with WebObjects, the learning curve can be a little steep.

There are an increasing number of open source projects using Tapestry which is making it a lot easier to get started by looking at code examples.  Here are a few worth checking out:

  • Wookie – Collaborative book editing.
  • Tapestry Hotel Booking – An example application that shows how to use common Tapestry features with live demo.
  • Tapestry Jumpstart – A basic application setup along with demonstrations of a bunch of common components. This is very useful as a reference.

Keep From Giving Away Passwords Like Gawker


Some time ago I made a comment on In order to comment I had to create a login.  I used an email account that I mainly use for junk email and a simple password.  In the past week, some hackers got into Gawker’s servers and downloaded among other things the user database.

When you store a password in a database, you are supposed to hash it.  In theory this turns your password into a random string so no one who has access to the database can actually see the password.  To verify your login, the server takes the password you give it and runs it through the same hash software.  If it matches what is stored in the database then you are given access.

The hash function is one way.  You can easily create the hashed value from the password, but it is difficult or impossible to create the password from the hashed value.  However, if you have a list of the hashes from a bunch of passwords, it is easy to search for a hash and then find its corresponding password.

To prove how great they were the hackers did this to nearly 200,000 accounts and posted it online.  These accounts were listed with their username, password and email address.  Mine was one of them.  Fortunately the password is something I haven’t used for several years and I haven’t used it for anything important for about a decade.  Still it is a bit unnerving to see my password out there for everyone to read.

I traced down a few places where the password was still being used–on a bookmark site and another comment account and changed those.  Fortunately I have been using 1Password for a few years so it was easy to do a search and find the accounts that were still using this password.

More recently I have been using 1Password to create a random password for each site I visit.  This is ideal because a long random password is going to be much more difficult to search for unless they have an enumerated list of every possible combination of keyboard characters of any length. Further, even if someone was able to find the random password that corresponds to the hash, it would only give them access to the account on one website–which they probably had to have access to in order to get to the database in the first place.

Salting Passwords

One thing Gawker/Lifehacker should have done is “salted” the passwords.  This would simply be adding some characters to the beginning or end of each password before running it through the hash function.  When you go to login again, the server simply takes the password you give and adds the same characters to it before running the hash on it again.  This makes the hashes much more difficult to find because it helps make sure that they aren’t common words.  If the hackers don’t know what the salt is or how it is applied, this should make it pretty much impossible to figure out the passwords.

However, in this case, the hackers got access to the Gawker sourcecode as well, so they would have known exactly how any salt was applied before making the hash. The hackers could go through and create hashes for a bunch of common words with the salt characters added to it, but that means they can’t rely on existing databases of hash to password mappings–they would have to calculate everything from scratch.  Still this is possible and and not unreasonable–particularly because they would only need to run through a bunch of words once in order to see if any user had used that as a password.

So if you have 1,000,000 users, running such a process against 1,000,000 common passwords will probably give you access to a number of the accounts.  That would require 1,000,000 hash operations using the dictionary file. I did a test and found that 1,000,000 MD5 hashes takes about 25 minutes on my i7 MacBook Pro.  This is just using the command line and a shell script.  I’m guessing that it could be done much faster using a different method, running it on a bigger machine or dividing it up among different computers.

Things can be made much more secure by setting things up so the password to hash mappings can’t be used to search the whole dataset. You want the hackers to need to recalculate the hashes for the entire dictionary file for each user.  This can be done if each password in hashed using a different salt value for each user’s password.

Different Salt Values

This could be as simple as merging the username and passwords together and then creating the password hash from that. This means that in order to compromise the same number of accounts as could be done with the same salt value would now require 1,000,000 X 1,000,000 hash operations.  This would mean significantly more time and/or more machines to run it against all of the passwords. It would take my computer somewhere in the 50 year range to do this.  Still if you got a cluster of 100 computers that are all 10x faster than my laptop, it could be done in around 400 hours.  So it is definitely doable, but it is starting to get much more expensive. Amazon’s high end EC2 instances cost $0.50 to $2.10 per hour so we are looking at costs in the $20,000 or higher range.

Still this isn’t prohibitive–particularly if the hackers are willing to use a smaller dictionary of passwords.  With at 100,000 dictionary of potential passwords the cost on Amazon falls into the $2,000 or higher range.

Hashing the Username or Email Address

Another idea would be to hash the username or email address. Usernames are going to be a bit more difficult to attack using a dictionary because they tend to be more unique.  In fact, if you were to combine the username and the password into the same hash it would be significantly more difficult because you couldn’t try to break the username or password by itself.  So something like:

hash(username + password)

Of course this would only work if you didn’t need the username anywhere else.  For example, you couldn’t use it to show the name of the person who left a comment.  This might work out ok if your application uses a username to login and a “screen name” to show to other users.

The problem with this is that you couldn’t look up a user without having their correct username and password.  This could be problematic for doing any type of password recovery. You might be able to work around this by using a third field that can uniquely identify the user–like an email address.

If your system doesn’t need to send emails, hashing the email address might be a better solution because these tend to be much longer and are automatically unique.  You could still send emails for password recovery because when given an email address you could find the row with a matching hash value.

If the average email address is 20 characters and can contain A-Z, @ and a period, then you’d be looking at 20^28 different combinations for a dictionary attack.  Actually the numbers would be significantly reduced if you took into consideration the fact that most email addresses end in .com, there is only one @ and other common patterns.  This is partially offset by the fact that some email addresses are longer than 20 characters and the fact that many people have username portions of their email that are more complicated than their passwords.

Something like the hash shown below would be impossible to break on any large scale, but wouldn’t allow traditional password recovery using email.

hash(emailaddress + password)

If someone forgot their password, it would be impossible to lookup their account via email using this approach.  Still, in some situations that might be acceptable–particularly if an alternative form of password recovery could be done either in person, SMS or some other method.

Log in