Robert's Blog


Thursday, February 28, 2008

A DB2 Person in an Agile Software Development Discussion

I'm posting this week from Orlando, Florida, USA, site of the winter meeting of the SHARE user group (www.share.org). I came here primarily for the purpose of delivering a presentation at the conference (the topic was SOA for DB2 people), but when I attend a conference as a speaker I also try to sit in on some presentations delivered by others (never stop learning, I say). Thus it was that I made my way yesterday afternoon to a session on agile software development. As it turned out, the speaker couldn't attend the conference due to a last-minute travel-related problem. I was thinking that we'd just walk, but the session moderator asked us if we wanted to spend the hour in an informal "free-for-all" discussion centered on agile development. Most of the people in the room liked that idea, and the ensuing discussion actually turned out to be, for me, one the highlights of my day at the conference.


Most of the people in the room were, as you might imagine, software developers. I was the only database person there, and I fessed up to that early on. We had some interesting conversations about the ways in which database people can help their coding colleagues who are wanting to do their thing in an agile way. I'm not an expert on the subject, but I'd say that agile development is largely about iterative development of an application, with lots of smaller-scale releases versus one or very few great big releases. It's also characterized by a cross-functional team-based approach to development, with differently-skilled people (not just programmers, but folks with specialized skills in testing, data, requirements analysis, etc., as well) working together to get each iterative release ready within the defined (usually pretty short) time frame (a good overview of agile software development can be found in wikipedia, at www.wikipedia.org). With respect to the role of data people in an agile development environment, this is what I heard from programmers during our impromptu "birds-of-a-feather" session:
  • Database people are very much welcome at the table. All too often, an organization's database specialists are not involved up-front in the development of new applications, and this is not just a matter of application programmers and architects not issuing invitations. Frequently, it's DBAs themselves who pass on opportunities ("not my job"), or it's an IT organizational structure that has DBAs working only to put together development databases in response to requirements that are "thrown over the transom" by programming groups. Sometimes a DBA has to take it upon himself (or herself) to get engaged early on with developers working on an agile application development project, but if that initiative is taken, chances are the programmers and architects will be very pleasantly surprised.
  • Speed is of the essence. The programmers might need a data store, but what they want is something done in a hurry. The database developed to facilitate agile development of an application doesn't have to be designed to the nth degree with the production environment in mind. Get something out there fast, and make schema changes (or what have you) as you deem advisable as you go along. Asking the developers to hold off on writing code until a really rigorous logical and physical database design is completed is probably not going to cut it. That's the heavy, old way of doing things.
  • Schema abstraction is very much appreciated. The developers want a database, but they'd rather not have to know about the underlying schema. There are various ways to abstract the application-database interface. One way is to package the SQL in server-side stored procedures. These can be called by all kinds of programs in all kinds of environments (definitely including Java and .NET), and to the calling programs DB2 stored procedures don't look any different from stored procedures in other DBMS environments. An even better step might be to expose the stored procedures as Web services (note that this can be done quite readily using IBM's new Data Studio product, which can be downloaded for free if you're going to use it for development purposes - see http://www-306.ibm.com/software/data/studio/). In any case, abstraction is your friend, too, because it enables you to make those database design changes you want to make (perhaps to improve application performance) in a behind-the-scenes way, without changing the interface to the data as the programmers see it.
Some of the other interesting comments made by session participants had to do with the CULTURAL change needed within many organizations to support an agile programming environment. You see, the traditional application development process has people working in a highly specialized way as though they were standing by an assembly line in a manufacturing plant. You know, coders code, database people design databases, testers test, and so on, but instead of working in cross-functional teams they man their posts and work on the stuff that comes their way. It's like put on the tires, put on the tires, put one the tires, etc. in a car factory. The work that goes on ahead of what you do (up the assembly line) and after what you do (down the line) is not your concern. You just make sure that the tires go on right. That approach may create comfort zones for some people ("put the tires on right and you'll be OK"), but it can lead to tunnel vision and to communication that flows too much in a vertical way ("I just do what my manager tells me to do") and too little in a horizontal way. People feel less personally invested in the finished product, and that can result in people not performing up to their potential since breakthrough results often rely in large part on the degree to which people feel an emotional attachment to what they do (i.e., when work really means something to people, work quality and productivity can soar).

The cultural change that may have to precede the establishment of a successful agile software development environment may be analogous to the revolutionary cross-functional team approach to automobile manufacturing introduced by Japanese car companies two or three decades ago. The face-to-face work done by cohesive groups of differently-skilled people engaged in the start-to-finish building of automobiles had a lot to do with the quantum leaps in quality, efficiency, and innovation seen in those factories. It set the standard that the rest of the auto-manufacturing world subsequently had to work towards. Similarly, agile development can be hindered, if not outright emasculated, by an emphasis within IT organizations on hierarchy, turf definition and protection, and work silo-ization. Of course, big-time organizational change has to be led from the top, but that can be challenging because many at the top are products of the legacy way of doing things. In large part, change may come down to trust: can technical professionals be trusted to perform as needed in roles that are less rigid and less precisely defined than the historical norm, with more floor-level leadership and initiative and less in the way of management-imposed structure? Personally, I believe that the rewards to an organization that can come from extending that kind of trust to the IT tech folks - from giving cross-functional teams an end goal and setting them lose to achieve it using their own creativity and drive - can be enormous.

Not a bad way to spend an hour at a conference, eh?

4 Comments:

Anonymous Anonymous said...

Good article. I have the following observations with the Agile development process:-

1. Teams are supposed to self-manage themselves. This sounds good in theory but as nobody's responsible for enforcing discipline, the opportunity for the more cynical to slack off is obvious.

2. No team member has a defined role. I believe Agile expects each team member to be a "generalizing specialist" i.e. they are supposed to know a bit about all of the roles required by the team but with specialities in some areas. I thought the economic benefits of specialist division of labour had been proved about 300 years ago. At it's extreme form it would take 1 man more than a year to produce 1 pin, but 10 people could produce 1000s of pins a year, if they each do their own part.

3. It's difficult to understand how the self-managing teams reward their members. Each member isn't equal so should be rewarded differently. Member's should be rewarded based on innate qualities such as intelligence & conscientiousness plus experience. Self-managed teams without roles sounds like a commune with all the inherent problems that history has shown are associated with communism. i.e. there's no reward for working hard, so why work hard.

4. Agile only seems applicable to software development projects. How would the bright move out from the commune in to management?

5. How do the agile software development teams interact with the rest of the company. You give the example of a DBA assisting in an application project. However, there are lots of part of the business for which iterative development isn't applicable e.g. systems, enterprise architecture, management reporting. In these environments central planning and stability obviously produce more desirable results.

March 8, 2008 at 6:00 AM  
Blogger Robert said...

You've provided some good food for thought. It certainly appears that one has to balance the desire for flexibility and innovation with the need for accountability. It could be that agile development is best suited to teams comprised largely of experienced professionals who have less need of oversight than people who are more new to the game.

March 8, 2008 at 6:33 PM  
Anonymous Anonymous said...

Exposing a stored-procedure as a web-service. THINK & REALIZE before you decide on that: how is the output represented?
Here at my client's site sometimes as output parameters, or as a result set or a combination of both! It should be obvious that only output parameters can easely be represented in a XML string containing the answer.
The same is an "issue" here: the stored procedures are written in cobol and also cobol programs are calling them. For performance reasons you might want to change the SQL CALL to a standard cobol-call. Again, this can only be done when the result is NOT a set.

March 11, 2008 at 3:58 AM  
Blogger Robert said...

Actually, if you check out the IBM Information Center material on Data Studio (http://publib.boulder.ibm.com/infocenter/dstudio/v1r1m0/index.jsp), you'll see that stored procedures that return data in result sets can indeed be exposed as Web services; furthermore, the same material describes how default XML schemas can be generated by Data Studio for stored procedures that return result sets.

March 12, 2008 at 6:13 PM  

Post a Comment

Subscribe to Post Comments [Atom]

<< Home