DB2 Notes from Warsaw (Day 3)

I'm blogging about my Day Three of the 2008 International DB2 Users Group European Conference on the morning of Day Four. Yesterday (Wednesday) I delivered my presentation (on what I call ultra-availability). Before that late-afternoon session, I was focused on business intelligence and data warehousing (much of the consulting work I've done over the past several months has been in this area). The day ended with a DB2 25th birthday party sponsored by IBM and held at an old manor house in the country just outside of Warsaw (the long bus ride back to the hotel proved to be a bonus, as I'll explain later).

Some of the take-aways from my Day Three:

• BI is a hot topic in DB2-land. I participated in the Business Intelligence and Data Warehousing Special Interest Group session (a SIG, in IDUG-ese - basically a "birds of a feather" get-together). The discussion was lively and covered a lot of ground. One of the IBMers present reminded people of the special DB2 for z/OS pricing (called DB2 for z/OS Value Unit Edition) available for qualifying BI workloads. The benefits and costs of ETL (extract/transform/load processes) were kicked around, with references to the need (or not) to aggregate data values bound for the data warehouse, differences between operational system and BI system database designs, and the potential integration of ETL with an organization's overall information lifecycle strategy (the latter brought up by consultant Jan Henderyckx, a reliably forward-thinking individual). The need for BI-supporting DB2 people to really understand what business users are trying to accomplish with a data warehouse was emphasized. One of the DB2 for z/OS participants spoke of the performance breakthroughs that can be achieved through the use of materialized query tables (MQT support, a feature well-known by DB2 for LUW people, was delivered on the mainframe platform with DB2 Version 8). Near the end of the discussion, participants talked about the growing popularity of "operational BI", especially on the system Z platform - a trend driven users' desire to access detailed as well as aggregated data records, and the importance of "data immediacy" (time-proximity to data change events).

• The latest on data warehousing and DB2 for z/OS. Willie Favero, one of IBM's top DB2 experts, delivered an excellent session on data warehousing and DB2 for z/OS. He started out by showing text from IBM's 1983 announcement of DB2 which positioned the DBMS as an excellent foundation for decision-support applications (this was before the term "data warehouse" emerged as a label for BI database systems). Willie noted that two of the factors driving an increase in data warehouse activity on the mainframe DB2 platform are SLAs that match those for operational database systems (and so call for maximum uptime, a strength of DB2 and system Z), and ever-growing numbers of concurrent queries accessing data warehouses (the z/OS operating system is very good at managing very large numbers of concurrently active tasks). A recently completed performance benchmark run at IBM's lab in Poughkeepsie, New York, was described: a 50-terabyte database, with a 300 billion-row table, and 200-300 query-issuing clients hitting the warehouse at one time. Willie was very pumped up about the results of the benchmark, which will be documented in an IBM "red book" that should be out within the next couple of months. An interesting slide in Willie's presentation showed the evolution of decision support applications from those that used query and reporting to gain insight into what happened, to deep analysis to try to better predict what will happen, to the current leading-edge systems that enable analysis of "right now" events so as to see more clearly things that are happening now. The benefits of DB2 for z/OS hardware-assisted compression were covered: 50% space savings, on average (with savings of 80% or more seen for some tables), with virtually no overhead on read (uncompress) operations. A recent enhancement: starting with DB2 for z/OS V8, the 64 KB compression dictionary that goes with each compressed data set (each partition of a compressed partitioned tablespace has one) is stored "above the bar" (i.e., above the 2 GB level) in the DB2 address space. Willie wrapped up with some discussion of DB2 for z/OS query parallelism: 1) use it, because it works great, 2) let DB2 determine the degree of parallelism for queries, because it does that very well, 3) remember that sysplex query parallelism can deliver massive parallel processing by splitting a query across multiple DB2 subsystems in a data sharing group, and 4) remember that query parallelism is one of the best ways to drive usage of a zIIP processor (a specialized mainframe engine that can offload certain types of work from the general-purpose CPUs but which does not affect mainframe software pricing).

• People are asking the right questions about high availability. I delivered a presentation on ultra-availability, pushing people to think beyond merely "great" availability to "ultimate" availability, which might be described as "never down, never lose anything, even in a disaster recovery situation." In discussions with attendees after the session, I heard a lot about the costs - hardware, software, programming - associated with getting closer and closer to ultimate availability for a data-serving system. Super-high availability can indeed be a pricey proposition, but IT people can serve their organizations by developing and costing solutions for ultra-availability (versus settling for less by assuming that the organization won't commit to achieving an audacious availability goal) and letting upper management make the call as to whether or not the potential payoff is worth the required investment.

• You have to love a line of work that can stay fun over a 5-decade career. On the bus ride back to the Hilton following the aforementioned DB2 25th anniversary party, I had the pleasure of sitting next to Tor Stryker (I hope that I have the spelling right), a lead data architect for one of Norway's top insurance companies. Tor is in his fifth decade in IT (all with the same company), and he still gets a tremendous kick out of helping to advance his organization's IT capabilities. He spoke of the days, way back, when programmers had to write code that would move sections of their own programs in and out of server memory because the whole thing couldn't fit within the few kilobytes of available space. Recently, he's helped to extend the functionality of legacy CICS-DB2 programs to Java-based client systems by enabling them to be accessed via DB2 stored procedure calls, and he has been a leader in the development of rules-based applications that can be extended, functionality-wise, much more quickly than old-style monolithic applications, thereby enhancing his company's operational agility for competitive advantage. All of this information was delivered with smiles and enthusiasm that were inspiring. Getting paid to learn and to innovate is indeed a good thing, Tor. Thanks for the reminder.