Way back in 2001, I worked with Russ Gold (founder/maintainer of HttpUnit and otherwise great guy) on the HP (formerly Bluestone) Application Server 8.0. A J2EE 1.3 server we collectively built in less than a year. That project was one of my best professional experiences and also a time where I got to know many other first-rate software folks, some of whom I continue to collaborate with to this day. Bruce Kratz is now at PrincetonSoftech, along with Al Smith, Pete Petersen, and Jay Hiremath. I worked very closely with Jon Maron and Mark Little and after we split from HP, we kept together to write a book. Jon and I still work together and Mark is at Red Hat. In looking for some info on agile programming that I need for a forthcoming blog entry, I stumbled on this article from Google cache profiling some of the work Russ and I did on HPAS. I had fun reading it and thought I'd save it from the bit bucket for a few more years...
[edit: after the initial post, I found an article Russ and I had written for the HP middleware newsletter. That article is included at the end of the post as well.]
XP's 12-practice methodology may play an important role for mobile developers struggling to maintain footing in a landscape evolving at warp speeds. It's already working For HP Middleware teams.
By Amy Cowen
Picture of Two Coders at a Computer
Agile or extreme?
While "extreme programming" remains popular in today's coding circles, a newer catchall phrase frequently appears on the scene - agile programming. In documenting their foray into XP, Gold and Pavlik use agile programming as an umbrella of sorts for a group of lightweight software development methodologies which includes XP.
"There was the recognition that there are a number of other methodologies which share XP's general approach of adapt rather than anticipate," explains Gold, addressing a growing shift from "extreme" to "agile" programming.
"Then too, at least part of the shift was probably Microsoft's fault," Gold continues. "By adopting the term XP for their new OS, they muddied the mindscape somewhat. Of course, now that Microsoft has adopted the slogan regarding agile businesses..."
Others attribute the shift from "extreme" to "agile" to be one of image. "Agile" conjures up speed and flexibility - but not risk - a change which could prove less intimidating for managers deciding whether or not to give an alternative lightweight methodology a spin.
Extreme Programming... Just the phrase raises images associated with sports from the X Games, sports that hover on the edge of mainstream legitimacy - dangerous, radical, and rule-breaking. Indeed, XP's name imbues it with a heady aura of defiance, post-modernity, and the spirit of daredevils and renegades.
Reviewing the tenets of XP, it becomes clear that XP is extreme in its level of difference from other traditional software methodologies. The irony of XP, however, is that it breaks the rules first and foremost by initiating a new set of rules.
And playing by the rules is key in XP.
This is not free-for-all programming where anything goes, where radical features are dreamed up and coded by pizza chugging hacksters who thought of a cool tweak while skateboarding through the office cubicles on a midnight break.
XP does encourage turning things up a notch. But it does so by offering a set of rules designed to ensure successful communication between team members, encourage a three musketeers mentality, rev up the production process, produce clearer and more straightforward and stable code, streamline costs, and, ultimately, enable teams to deliver solid products to happy clients.
For developers working in the ever-changing mobile application space, an "extreme" approach that churns out proudcts at light speeds compared to traditional development cycles might give developers an edge and put apps on the market in time to meet consumer demand - in time to nourish a growing user base and win critical marketshare.
XP on the scene
XP was developed and first documented by Kent Beck in his 1996 Extreme Programming Explained. Under Beck's guidance the first XP implementation happened on the now legendary - in XP circles - Chrysler Comprehensive Compensation - or C3 - development team.
Since its automotive beginnings, XP has infiltrated the ranks in all venues of software development, with success stories cropping up at companies large and small.
Foundations to build on
Beck bases the XP methodology on four core values - communication, simplicity, feedback, and courage. These values underwrite the 12 guiding XP practices (see sidebar below) and provide touchstones visible throughout the development cycle.
Traditional development often involves teams of coders holed up in individual cubbies, each working on pieces of a larger project. When finished, the pieces are assembled, and beta testing begins to see what works and what doesn't.
Assuming each coder has her own way of writing the code, her own way of approaching problems, and minimal communication between team members, it is easy to see how problems are likely to appear when the work of such individual contributors gets glued together.
There's not much holding the "whole" together.
For an XP team, however, the "whole" is being considered throughout.
The 12 XP Practices: The Planning Process, Small Releases, Metaphor, Simple Design, Testing, Refactoring, Pair Programming, Collective Ownership, Continuous Integration, 40-Hour Week, On-Site Customer, Coding Standard
From collective ownership of the code to the emphasis on the simplest design and a commitment to refactoring, the 12 practices that form the foundation of the XP methodology formalize the development process, bringing high levels of discipline, communication, and team spirit to an arena typically ruled by solo code jockeys.
HP goes XP
For Bruce Kratz, HP Middleware Lab Director, XP has found firm footing with his teams. Kratz chalks the HP Middleware team's adoption of XP methodologies up to rapidly changing needs.
"In our industry, requirements change every 90 - 120 days," explains Kratz. "XP allows us to release product in short intervals with a high level of confidence that it is a quality product."
The Middleware Division's first XP project was conducted by the HP Application Server (HPAS) team when working on Version 8 of the server. Following the successful use of XP for HPAS, both the Rich Media Technology team and the Mobile Infrastructure team are working with XP methodologies, says Kratz.
Greg Pavlik and Russell Gold, architects in the Middleware Division, worked on rebuilding EJB-related code for HP Application Server 8 and have documented their team's use of XP. While they didn't employ all 12 of XP's practices on the project, both found working with XP a positive and successful experience.
According to Pavlik, XP was the right approach for the project because "we needed a process that was highly resilient to change and evolution in the code base."
"[HP has] been in the application server business" since its acquisition of Bluestone, continues Pavlik. "But the code base today is completely different than what we started with, so the rate of change is phenomenal. Some of the software has been rewritten from scratch, but the EJB 1.1 server had been battle-hardened in some of the container logic. We wanted to preserve the foundations that we knew were solid, but didn't want them to become an albatross either. You have to have a process that supports constant and dynamic refactoring to move forward with new features and architecture changes without turning a project into spaghetti code."
The realities of ongoing and near-constant change are ones developers constantly battle.
XP helps offset the problems - and costs - associated with such change, making it a solid strategy for the HPAS team.
"You'll notice that the first XP book by Kent Beck was subtitled Embrace Change, and in one sense that's what XP is really about - flattening the cost of change away from an exponential growth curve," continues Pavlik.
Simple is as simple does
Simplicity may be a surprising tenet to see in XP discussions, but on an XP project, "simple" is the result of hard, calculated, and relentless work. Striving towards the simplest design, Beck says the XP coach has to ask, "what is the simplest thing that could possibly work?"
The simplest solution is often one that turns a blind eye to the future, and this is important in XP because XP discourages the integration of hooks for future expansion and development. Instead, on an XP project, the goal is to write a program that solves the client's immediate needs.
"Extreme Programming (XP) is a high-discipline, low-ceremony approach to development, meaning that it does not produce a lot of formal documentation or rely on a lot of formal reviews, but does insist that its practitioners be consistent in following the essential practices."
- Pavlik and Gold
"Simplicity is not easy," says Beck. "It is the hardest thing in the world to not look toward the things you'll need to implement tomorrow and next week and next month. But compulsively thinking ahead is listening to the fear of the exponential cost of change curve."
According to Beck, the "right" design is one that:
1. Runs all the tests.
2. Has no duplicated logic.
3. States every intention important to the programmers.
4. Has the fewest possible classes and methods.
Skeptics might view this focus on immediate needs - rather than on the "big picture" of the application over time - as a potentially costly approach since the code isn't necessarily extensible in ways that make it easily upgradable as needs evolve - a reality which could mean having to start over at the beginning when new requirements take center stage.
However, for those working with XP, the benefits to the XP focus on simplicity outweigh the risks.
"I feel over-engineering for the future, in anticipation of features that are not yet needed, winds up costing more," says Kratz.
Show and tell
Key to accelerating the development cycle and time-to-market is XP's focus on frequent releases and ongoing integration. Small 'nuts-and-bolts' releases happen throughout the development. The customer can see the product, evaluate the current feature set, and consider new features he wants to incorporate or add.
The scope of the project evolves and changes with each micro-release.
In part, the theory is that the penultimate product reflects what he wants rather than what he "thought" he wanted at the outset - leading to happier customers and ensuring the coding throughout the project was appropriately targeted at the desired features.
The lack of a fully-determined development specification - and the determination of features on an 'as we go' basis may sound shortsighted - and costly - to XP-skeptics.
Developers who have worked on projects in which the customer wasn't sophisticated enough to know up front what he needed, what was possible, and what would work best, may find the process of shedding reams of documentation, outlines, feature specs, and flowcharts intimidating if not downright foolish. For these developers - bitten by the clueless client in the past - XP may seem to set up a never-ending - rather than faster - project.
But for those who have gone the XP route, such fears seem misplaced.
A traditional development cycle can be so long - and so bogged down in paperwork - that by the end of the project the scope of work is no longer fully adequate to meet current client or market needs.
As Kratz notes, "No matter if we release frequently or [in] longer cycles, customer feedback may launch us in a new direction. By releasing frequently, we have the opportunity to get this feedback sooner and incorporate it into the next release."
"The Application Server Market product space is constantly evolving," Kratz adds. "So we expect to be in a constant development cycle anyway. XP allows us to keep up with technology and measure our success against customers and the market."
Back-seat driver
One of the most talked-about hallmarks of XP is the practice of pair programming. Programmers team up and work together when writing all code. One partner sits at the computer and does the typing. The other member of the pair watches everything that is typed in - looking for problems, mistakes, and potential pitfalls, and helping to talk through the code as it is being written.
While it may sound costly to devote two programmers to the same task, the XP theory is that working in a pair produces cleaner and better code, thus reducing time spent later debugging and cleaning up.
Working with someone else can be a big adjustment since programmers traditionally tend to work solo - disappearing for hours or even days to bang out some code to solve a certain problem.
"There's an important watchdog role to play when you're not the coder in the pair, but it's not what people are accustomed to. The temptation for me [when working solo] is always there to disappear for a few days and come back with a subsystem coded, missing tests, etc. Pairing can help add discipline."
- Greg Pavlik
Working in a pair means having someone else constantly watching and evaluating the code. The pairs are also constantly in communication. Some XP advocates go so far as to say that pairs should be talking at least every 45 seconds or so.
Despite the adjustment, pair programming seems to be a hit among the teams that have used it.
Having worked in an XP programming pair, Gold prefers the duo arrangement to working alone.
"I like working in a pair whenever possible," he says, "since I find it focuses me better and allows someone to catch me before I go down a path that in hindsight was foolish."
Pavlik is more noncommittal about whether he prefers to code solo or in a pair, but he readily admits that pair programming produces "much better code and ensures better practices." The caveat for Pavlik is that pair programming is slower than working alone, so not feasible for all projects.
For both Pavlik and Gold, identifying the hardest part of adjusting to pair programming is easy - "Not writing code all the time," says Pavlik.
Gold concurs, noting that the hardest aspect of working in a pair is "staying engaged when not doing the typing."
This is not necessarily surprising since for a programmer used to having free reign at the keyboard, sitting back and watching can make the fingers itch. Harnessing years of being able to tackle problems in one's own way and watch, instead, as another coder works can be difficult.
But careful watching - in addition to solid communication with the person in the driver's seat - makes pair programming work.
On the flip side, the person driving the code has to also adjust the way he works in order to remain in communication with the watcher.
"An experienced pair programmer helps his partner by thinking out loud while typing and explaining what he is doing at every point," explains Russell.
Clearly, pair programming dramatically alters how a developer approaches writing code regardless of whether or not the coder is sitting in front of the keyboard or watching from a hands-off position.
Pairs are dynamic, too, changing frequently as members of the team group and re-group to work together on aspects of the project.
Every pair will be different. Some pairs consist of two strong coders. Sometimes a pair will consist of a more experienced coder and a newer team member. Pairs might even be made up of designers and architects. Nevertheless, XP theorists maintain there are strengths in all kinds of teams - even a team of two new programmers can produce stronger code than if they were working independently.
Speaking from a managerial perspective, Kratz says he has been surprised at how well pair programming has worked within the Middleware teams.
"It's probably not for everyone," he acknowledges, "but from my seat, things are thought out better, younger team members are becoming stronger quicker, and things are getting done. Hey, they may even be having fun!"
Goodbye paper trail
An interesting offshoot of XP is a reduction in documentation requirements. Team members are in constant communication with each other - and in frequent communication with the customer - so there is no reason to spend valuable time producing documentation when there is coding that can be done instead.
In XP, the focus is on producing code - not producing paper that talks about the code that will someday be produced. The time saved leaves coders fresh to tackle the project and works with the other XP practices to push the project towards early completion.
For those with either a packrat or a cover-our-tail mentality, putting aside the documentation process can feel both risky and maybe even leave the process feeling naked.
But XP veterans have found the reduced documentation to be a liberating aspect of the process.
"Too many design docs mean death for a project," says Pavlik. "When people talk about project failure and 'analysis paralysis,' they're talking about things like a giant static model, reams of documentation, and no working code or functionality."
With a smile, Russell agrees that it wasn't hard to ditch documentation.
"In my experience, most development projects don't actually produce a lot of usable documentation during development anyway," he notes. "Most design documentation built at the start of the process winds up obsolete by the end and is not maintained."
XP recognizes this and emphasizes writing code that is clear enough to serve as its own documentation. XP advocates add that the increased emphasis on person-to-person communication helps replace the need for a paper trail.
"The biggest hurdle is teaching developers to refactor working code," says Russell.
Refactor, refactor, refactor
A central XP practice is relentless refactoring. From the outside, the process of refactoring can seem a bit vague - coders review their code to ensure that they've taken the most straightforward, most streamlined, and simplest approach. All excess is trimmed or tweaked, keeping the code facile, lean, and clean.
In their own account of their experience using XP, Gold and Pavlik write:
"It is very common to develop software which works and fulfills all existing requirements - but which cannot be easily extended to handle new requirements. The most common reaction to encountering such a situation is to sigh, roll up one's sleeves, and rewrite the offending code, taking into consideration the new needs. But doing so discards all of the work that went into debugging the original code."
XP's emphasis on refactoring counters this approach, leaving code in such a streamlined state "that new features can be added easily without changing the behavior of the working code at all," explain Gold and Pavlik.
According to Pavlik, "Refactoring is really the core practice."
Explaining the important role refactoring plays, Gold says, "The essence is to recognize that there are a large number of well understood incremental transforms which you can make to software to improve its organization and design. By applying them, one after another, and verifying that you have not made a mistake by rerunning your unit tests, you can safely restructure your software to minimize coupling, increase cohesion, shrink your methods until they can be easily understood, and give your methods, classes, and variables names which clearly convey their meaning."
Boot camp for extreme developers
Join other extreme coders in the mobile space in Helsinki, Finland, April 9-11 for 60 hours of non-stop coding and development. As you squeeze nearly 2 weeks of work into a few days, you can enable your mobile e-service app with hp mobile technologies.
Sleeping bags, saunas, and plenty of food will be available. So bring your app, your imagination and mobile services vision, and get extreme!
Visit the HP Bazaar Site for more information or to register.
"The most important aspect is to change the system without affecting the behavior," adds Pavlik. "If you are changing the behavior and the structure of system elements, you are not refactoring."
Producing leaner, more malleable code, refactoring addresses what otherwise might prove a shortcoming in the XP approach - XP's refusal to code for the future. By insisting the final code is solid, pared down so that only the mandatory essence remains, XP leaves code in a state where future work can begin with minimal rebuilding.
Indeed, refactoring "is the mechanism by which you are able to affect the cost of change curve," says Pavlik.
(For more information on refactoring, both Gold and Pavlik encourage looking at Refactoring by Martin Fowler. "It is one of those books which every serious developer should own," says Gold.)
"Testing, testing"
Another important XP practice is the use of "tests" as a QA tool. This may not sound unusual. But in XP, the tests are written prior to the commencement of writing the code. The tests are then deployed throughout to ensure quality control and to highlight bugs that might crop up.
While at first glance it seems convoluted to create tests before what's even being built has been concretely defined - keeping in mind that XP takes an "add as you go" approach to determining the scope, extreme coders find the process of creating the tests clarifying.
According to Pavlik and Gold, "testing was one of our clear wins on this project. We got into the habit of writing unit tests for the software, using a combination of ant, JUnit, and HttpUnit to run the tests in suites. We adopted a rule that nobody could commit changes to the baseline unless all tests passed. As a result, the EJB container code was almost never broken by something one the of the EJB developers did, and we had a high degree of confidence that something that worked at one time continued to work through the 8.0 release."
"Creating the tests first help you understand what you are attempting to build," Kratz adds.
A winning combination
For HP's Middleware Division, XP's 12 practices have already added up to measurable success.
However, while XP's tenets have held up well so far, Kratz remains realistic, admitting that XP isn't always the right approach.
"Smart teams will know when to follow the XP approach and when to resort to traditional development methodologies. My thought is that 'one size does not fit all.'"
Gold, too, remains pragmatic.
"XP is not magic. It is a set of 12 well-defined practices which have been found to work together well."
And the second article....
Experiences with Agile Programming Models on HP-AS 8.0
Background and Definitions
Among the constant challenges in the development of software is the problem of how to handle change, especially in system requirements. Most traditional models of development have noted that it is much cheaper to fix a problem during requirements or design than to correct it once the code has already been written. As a result, these models have tended to focus on trying to be as thorough as possible in making sure that the requirements are complete and correct, and that the design addresses all major considerations which might arise before moving on to implementation.
Unfortunately, in most systems of interest to J2EE developers the problems simply cannot be known completely until the system is built and tried. And of course, once this has happened, changing the system is difficult, costly, and error-prone.
An alternative approach for dealing with change is to build a system around what is known for certain initially, and to make the system adaptable throughout the project lifecycle using techniques that reduce the costs and risks associated with modifying working code. The use of such techniques is known as Agile Programming.
What is Agile Programming?
In the last three years or so there has been a lot of discussion among software vendors about agile software development processes. In particular, there is a lot of excitement associated with the practices that Kent Beck popularized in his book Extreme Programming Explained: Embrace Change. Other "agile" processes that are gaining adherents in the developer community include systems of practices described in SCRUM, Feature Driven Software Development, and Pragmatic Programming. Each of these systems shares some common themes: close collaboration between developers and customers, recognition that changing requirements are a reality of software, and that development practices must play to the strengths of how developers work best.
We chose to use practices from Extreme Programming, probably the best known of the Agile Programming processes, to develop HP Application Server EJB container.
What is Extreme Programming?
Extreme Programming (XP) is a high-discipline, low-ceremony approach to development, meaning that it does not produce a lot of formal documentation or rely on a lot of formal reviews, but does insist that its practitioners be consistent in following the essential practices, including:
Onsite Customer The customer makes all functionality decisions and is always available
The Planning Game Customers get detailed continuous control over what work is done
System Metaphor The overall design is described in terms of shared concepts
Simple Design The design addresses current needs, not possible future additions
Collective Ownership All code is the responsibility of multiple developers
Forty-hour Week Developers may not work overtime two weeks in a row
Test-Driven Design Functionality is validated via automated tests written before coding
Pair Programming Code is written in pairs, thus giving continuous reviews
Continuous Integration Code is pushed to the baseline on a daily basis
Refactor Mercilessly Poorly structured code is cleaned up aggressively
Small Releases Customers are given updated working code to use frequently
Coding Conventions The developers agree on a common coding style and follow it
These practices have been shown to work together to produce consistent, reliable and flexible software without running a development team into the ground. Of course, XP is deeper than just a set of practices. It declares that programming should be a humane discipline that doesn't drive talent out of the trade.
And of course, as part of its "embrace change" attitude, XP insists that each team constantly analyze those practices it is following and change them when appropriate. It is more important to emphasis the values of listening, testing, coding, and designing than to adhere to someone else's checklist or rules.
EJB Development
Both of us worked on the EJB container in the latest release of HP Application Server. The project was a major undertaking. Within a six month time frame, the team had the responsibility of rearchitecting and rewriting most of the EJB-related code base. We needed to fit the container into a service framework on which the application server was being built, interact with other newly defined services, support a number of new features like hot redeployment and pluggable transport protocols (ensuring that all J2EE Compatibility Test Suite tests ran at 100%), and bring the container in line with the EJB 2.0 specification, public final draft two.
While we had experimented with XP techniques like automated unit testing in the past, we wanted to learn more about XP and what it had to offer through experience. Another great benefit was that Russ had joined the team at this point, and brought with him experience with XP. We'll try to summarize some of the what we did do, some of what we didn't do, and look at what the implications were for our project.
XP Programming Techniques We Used
Frequent Testing
Testing was one of our clear wins on this project. We got into the habit of writing unit tests for the software, using a combination of ant, JUnit, and HttpUnit to run the tests in suites. We adopted a rule that nobody could commit changes to the baseline unless all tests passed. As a result, the EJB container code was almost never broken by something on the of the EJB developers did, and we had a high degree of confidence that something that worked at one time continued to work through the 8.0 release.
One of the most useful rules the team developed was that bug reports should be accompanied by automated JUnit tests. The test code showed the problem in a repeatable way and the tests were added into the developer suite to ensure that regression didn't occur.
Continuous Integration
We encouraged our developers to check in their code every day, while keeping the baseline working (all regression tests running at 100%). This was tricky since we automatically absorbed all changes from other groups working on other portions of the application server every day as well. At times the upstream tests would miss something that would affect us further downstream. Since we couldn't always wait for the upstream changes to be fixed, this caused us to stumble a few times when we checked in code that broke the same tests. But for the most part, we were able to keep the baseline clean throughout the cycle, and this really helped us maintain the quality of the software and protect against regression even during aggressive overhauls of the container.
Another important benefit we saw from frequent integration was that it helped everyone understand how both refactoring and new development were effecting the evolution of the system. Since change was incremental, it was comprehensible. If we had all waited until some integration phase to bring pieces together within the system, everything that a developer wasn't actively working on would have become like a black box.
Refactor Mercilessly
It is very common to develop software which works and fulfills all existing requirements - but which cannot be easily extended to handle new requirements. The most common reaction to encountering such a situation is to sigh, roll up one's sleeves, and rewrite the offending code, taking into consideration the new needs. But doing so discards all of the work that went into debugging the original code; it's also time consuming, and often leads to code no better structured than the version which was replaced. Refactoring addresses this, providing a way to restructure the original code so that new features can be added easily without changing the behavior of the working code at all. This is one of the most important practices of XP; in some ways, it is the essence of XP. There are two things about refactoring that are critical to understand. First of all, it doesn't work without tests, period. If you don't have the tests, you will almost certainly break the software. Secondly, refactoring can be divided into small and well understood steps, catalogued in such places as Martin Fowler's Refactoring: Improving the Design of Existing Code and on his website, http://www.refactoring.com. It is also greatly simplified by support in automated tools -- something that's now creeping into Java IDEs.
There were a number of times during the development of the EJB container when we started with code left over from the Total-e-Server implementation, and refactored it to allow the addition of new features. We didn't do a lot of it, but what we did was invaluable. For example, we were able to refactor a legacy class from over 2500 lines to 300 lines, gradually, over the course of the release. This made it possible for us to more readily support hot redeployment of EJBs within the container.
Simple Design
Trusting in our ability to refactor as needed, we tried hard to avoid common expensive practices such as "adding hooks" for future growth. This allowed us to avoid unnecessary code, and we did not see any place where it hurt us. It also made the code much easier to understand.
User Stories
XP also promotes the creation of user stories, which are lightweight use cases for pieces of functionality. User stories were something we used consistently, but also something we struggled with doing correctly. The user stories we developed were very much like functional requirements for the system. As a result many of the stories were driven by developers rather than the customer proxy. We did, however, use these as a low-investment strategy for planning. Our group manager played the role of a tracker and tried to make sure the stories within a cycle were consistent with the velocity measurements he had taken for past cycles. This was a great benefit for tracking and planning with agility.
XP Programming Techniques We Didn't Use Consistently
Collective Ownership
With XP, assignments are distributed so that developers don't specialize. In practice, we assigned developers to specific functional areas and for the most part did not overlap, fearing that without specialization we would not be able to develop the necessary expertise in each area. The downside, of course, is that there are many parts of the code known by only one developer and never reviewed or even looked at by the others. It is too soon to tell how this will affect future maintenance.
Pair Programming
Pair programming is a concept that seems alien and we've found that this is an idea that people find hard to digest. For one thing, it's simply very different from what we're used to. It also seems counter-intuitive that a process designed around rapid software deliveries would tie two people to a computer. Turns out we were wrong.
We found that when we paired on user stories, different areas of experience of different programmers really combined to give the pair a deeper understanding of the problem space. As a general rule, pairing led to better code. We extended this outside of our project and attempted to always pair when we worked with other teams interfacing to our container. This helped both sides to get a better understanding of the critical glue code between subsystems.
Another critical point about pairing: it encourages discipline. If one person is behind and tempted to cheat on test coverage, for example, an individual can get away with it. Another benefit of pairing is that it tends to keep people focused. You can't waste time on nonproductive web surfing when someone is working with you.
When push came to shove, we needed to deliver and we fell back on what on the more conventional techniques that we knew. We had to rely on code reviews that were often stressful and grinding to audit each other, and the there were times when class design suffered. Pair programming may seem unnatural, but it has some real benefits that make it worth considering as a basic practice for your team.
Test-Driven Design
For the most part, the team didn't do "test first" programming. By the time we started seriously writing automated tests, most of the code was already in place, and needed updating rather than writing from scratch. In addition, despite reading the XP literature, we did not realize just how powerful this technique can be. Some of us did have the opportunity to write functionality that was brand-new for EJB 2.0, and we did use test-first in those parts. The result seemed to be fewer false starts and simpler designs than in areas built without such tests.
Architecture vs. Metaphor
XP puts forward the notion of a metaphor for a system as a substitute for architecture. We didn't attempt this: we spent time working out a macro-architecture, talking it through face-to-face, and adjusting it as necessary as we progressed. The metaphor idea didn't seem to make sense for something like a framework and the approach we took seemed to work well. So this was an aspect of XP that we didn't utilize, but also didn't feel that we were missing anything.
Limits of Tests
It also became quickly clear that there are limits to what the unit tests provide. Tests aren't a substitute for really digging into the system. Constant and liberal changes can result in subtle changes of untested assumptions. It's easy to say that the problem is there aren't enough tests which, while true, isn't an answer. After all, there will never be enough tests to check every assumption about the system. Software is complicated, and middleware particularly so: it combines the challenges of concurrent, distributed systems programming with a need for an extensive understanding of low-level concepts in security, distributed transaction processing, high availability system engineering, etc.
Performance
One of the questions we had was how XP would effect performance. Refactoring typically leads to more indirection and we specifically avoided writing "optimized" code. Instead, we waited until after all features were complete to systematically profile the system under load with real end-user applications. This paid off hugely: the container was considerably faster than previous iterations and the unit tests helped to ensure that the optimizations we introduced did not break the system.
The most important bottleneck in middleware for distributed systems is resource contention, and we were able to isolate and eliminate the contention issues we identified. The result was not only a faster system, but also a more scaleable system. We had great success with this approach, but would hesitate to recommend it without unit tests in place.
Conclusion
Over the course of the release, there were times when even we doubted our ability to deliver on all of the requirements in the time required. It's our strong conclusion that using the techniques from XP that we used were fundamental to pulling the software together into a well-received, sophisticated application server. We found that the basic practices combine and play off each other to help make better software and that we missed out on some key benefits from practices we weren't able to implement. We've worked on projects where the code regresses over time and we're happy to report that the code base for the EJB container took a different path: it got better and better as the release progressed. There are unique challenges to every project: some teams are very large, some are very big, some distributed geographically.
We would encourage you to experiment with XP and other Agile Programming techniques and adopt them to your environment. They pay off in a big way: well written software that meets real requirements in short time frames.
More information regarding Extreme Programming is available at:
http://www.extremeprogramming.com;
and for more information regarding Agile Programming, check out:
http://www.agilealliance.org.
ABOUT THE AUTHORS:
This article was contributed by Russell Gold and Greg Pavlik architects with Hewlett-Packard's Middleware Division. Russell has been instrumental in applying the eXtreme programming methodologies to the Middleware development process. Russell is also the original author and maintainer of HttpUnit, an open source library for automating tests of web sites. Greg was the lead architect on the HP Application Server's EJB 2.0 and 1.1 implementations and has been a member of the EJB expert group.
No comments:
Post a Comment