Still looking for open source DB2 monitoring tools?

Hyperic website:Hyperic HQ open source version 3.2 released

It's fun when somebody crashes a dull party, and very few parties have been more dreary over the past few years than the gathering of open-source monitoring tools that work out of the box with DB2 LUW. After deploying and customizing Nagios for a few of my DB2 customers, I felt the need to work up a bit of a rationalization for its abstruse configuration files, but we all know how much comfort rationalizations offer. What I really needed was a solid monitoring stack for DB2, and -- even with a price tag of zero dollars American -- Nagios was often a difficult sell (although their upcoming Version 3 looks somewhat better).

When my friend Irving described Hyperic HQ as the monitoring platform that Nagios should be, I was more than ready to give it a spin. HQ's built-in support of DB2 V8 and DB2 9 was enough encouragement for me to get it running. Seeing so many servers and services automatically register themselves with the HQ server was a welcome change from the "don't do me any favors" philosophy of Nagios.

Before the shiny new release of Hyperic HQ 3.2 earlier this week, I poked and prodded versions 3.0.5 and 3.1.4 as they monitored DB2 and other servers running in my lab. I wasn't wild about the documentation's insistence that I monitor DB2 as the instance owner, so I made the DB2 plugin connect as a SYSMON user instead. The plugin did a decent job of autodetecting the tables and tablespaces that were encountering activity. Best of all, once I installed my agents, I was able to configure everything else over the web, with no hateful configuration files expecting me to learn an arcane language that makes httpd.conf look like a Little Golden Book. One more thing: just about everything in HQ can be graphed. Don't get me wrong, I'm still a big fan of RRDTool, but I'll happily use another graphing tool if it's going to do nearly all the work for me.

The secret gem of Hyperic HQ is the SQL Query server type, which will run any piece of SQL you can throw at it, and even graph the resulting number. If the number returned (or query execution time) fall outside of your predefined limits, it will send you a civilized email alert. Unless you'd rather wait for your users to inform you of database problems, I heartily recommend using Hyperic HQ to wire up DB2 (if not everything) and learn what normal looks like in your shop.

Hyperic HQ is an open source project written in J2EE that runs on an embedded JBoss server. Its internal database is PostgreSQL, but MySQL and Oracle are also supported. If anyone on Planet DB2 are interested in lending a hand with the DB2 port, I'm sure you'll receive a warm welcome from the HQ team.

Non-DB2: Someone blog-tagged me

Blog tag 8*8:My response to Craig Mullins

If you follow the Planet DB2 blog aggregator, you'll see that there's been some blog-tagging going on, with my blog being one of the more recent targets. Although I generally stick to DB2-related content for this blog, I'll play along and reveal eight things you may not know about me. Then I'll tag eight more bloggers before someone else gets to them (we know a lot of the same people).

1. I played the tenor saxophone pretty much every day from junior high through college, in nearly every possible format: marching band, concert band, jazz big band, combo, and various jazz/funk/R&B groups that gigged around town. Although my grades were good, all of my scholarships were music-related.

2. Being a band geek meant it was inevitable that I would meet my future wife at band camp. This year we will have been married ten years.

3. By the time I first rode on an airplane, I was 18 and had just finished a summer internship at Honeywell, where I was editing Pascal source code for the Airbus A320. So, I was debugging airliners before I ever set foot on one.

4. Since moving to Portland over 11 years ago, I've become a bit obsessed about coffee. I even volunteered (but did not compete) at a regional barista competition, which was more than enough to convince me to stick with my day job. After giving up on trying to make good espresso at home, I now brew my morning coffee a cup at a time, using a Melitta cone, electric teakettle, and two digital scales. (OCD much?)

5. Last spring I donated my 14-year-old car to charity and started riding a bicycle for the first time in 20 years. Despite all of my business travel last year, I still managed to bike over 500 miles on it, just running errands around town. It sure beats hunting for a parking space in downtown Portland.

6. I'm both proud and a bit surprised that the homegrown TiVo I built last summer from commodity PC hardware still works. It uses MythTV software to simultaneously record shows from two different HDTV channels. It's the envy of my neighbors (at least the ones who understand what it is).

7. A few years ago I found a really good recipe for baby back ribs, which I make every summer for my neighborhood potluck. Invite me over, and maybe I'll bring some.

8. 2008 marks my third year as an independent consultant, a bold and exciting move that I've never regretted. One of the things that has helped me succeed is having a large room to use exclusively as a home office. Working from home also gives me the opportunity to fiddle with various pieces of networking equipment, a secret hobby of mine.

That wasn't too painful. Now I take great pleasure in tagging Alexander, Dan, Dawn, Jeff, Leon, Martin, Scott, and Vincent

Free PHP software to monitor DB2

SourceForge.net website: DB2 Monitoring Console

If you're looking for a web-based monitoring suite for DB2 9, IBM's brand spanking new upload of DB2 Management Console (DMC) to SourceForge may be worth a look. Peter Kohlmann from IBM mentioned this project at IOD 2007, and it's good to see that DMC is already classified as Production/Stable, and free/open via the Apache 2 license. If you're not the type of person who builds and installs PHP servers for fun, Zend Core for IBM installed rather cleanly for me, allowing me to run DMC without much hassle. Give it a try and see how IBM is approaching the idea of building open-source, web-based monitoring for DB2.

Excellent photo of the worst movie theater ever courtesy of MadMask

DB2's latest TPC benchmark transforms into giant robot, beats up on other benchmarks

TPC website: 343,551 queries per hour on 10,000GB of TPC-H data (US$32.89 per QphH)

Perhaps one of the most surprising omissions at the IOD conference keynote sessions is any mention of IBM's October 15th TPC-H benchmark (the same day the conference began), in which they fired up a battalion of hardware and software to achieve fastest ten terabyte data warehouse benchmark of all time. Thirty-two POWER6 machines spinning a total of more than 3,000 disks served up 343,551 queries per hour, nearly double the speed of the previous throughput record (also set by DB2). Such surreal transaction rates also pushed the price-performance ratio to record lows for that category (well, at least by a couple of pennies, but we'll get to that later). However, with all benchmarks, there are quite a few things to consider.

As with most data warehousing benchmark submissions, the 3,000+ disks mentioned above are basically 90% empty. Remember how much pushback you got from your VP or CxO when you handed him or her a quote for the storage you needed? Now think of the look you'd get if you requested eleven times as much space purely for performance reasons. If anyone out there has been able to use their Jedi mind trick powers to pull off such a feat, please tell me how you did it. I promise I'll pass your name on to IBM and EMC so they can start a bidding war to hire you as the greatest storage sales rep of all time, and I will spend my referral bonus on a gloriously expensive coffeemaker designed by aerospace engineers.

Another curious aspect of these benchmarks is the gap between what IBM recommends and what they do in DB2 benchmarks. When it comes to the TPC-H data warehouse benchmark, IBM is not eating their own dog food. If you have attended a conference presentation or watched a webcast about DB2 9, you'll know that IBM is (justifiably) proud of several engine features that can profoundly improve performance:

• Self-tuning memory management


• Deep compression


• Multi-dimensional clustering


• Materialized query tables (Okay, this one is understandable, since the TPC have slapped so many restrictions on this area that it might as well be banned altogether)


• Block-based bufferpools (Also understandable, since there's little need to split up a bufferpool for a TPC-H workload that is geared exclusively toward block-based prefetching)

Were any of those features used in IBM's October 15th benchmark? Nope. Does IBM tell customers to exploit those features for their own data warehouses? All the time. The reason for this contradiction has to do primarily with database load time, a metric that is apparently much more significant in the bizarro universe of the TPC than in real-world shops, which rarely load a multi-terabyte warehouse from scratch. MDC tables would have chugged a bit to load so much unsorted data, and deep compression requires a table reorg (and a license for nearly 13,000 value units). What you end up with is a frustrating inconsistency between the real needs of data warehouse decision makers and a skewed technical experiment that purports to help people make those decisions.

That's not to say the benchmarks are pointless. If they were, I wouldn't waste time writing about them. Buried in those interesting details is IBM's preference to disable INTRA_PARALLEL in favor of running two DPF partitions per CPU core, resulting in 256 partitions that each manage barely 50GB of data. It's also not surprising that IBM applied table partitioning to the ORDERS and LINEITEM tables. Anyone running DB2 V8.2 Enterprise Edition should be scheming to figure out how to exploit this powerful feature as they plan an upgrade to DB2 9 or 9.5, since table partitioning is included in that edition at no extra charge.

The last part I wanted to bring up is the price, since the final price-performance ratio carries so much weight. IBM offered themselves a 48% discount for much of the hardware and software, but other competing vendors pull the same stunt with their seven and eight-figure TPC-H configurations, so don't get too worked up over it. Unless you plan to spend millions of dollars up front for such a system, you are unlikely to realize that deep a discount.

Overall, the numbers realized in this benchmark are good news. IBM took their hottest new UNIX hardware and proved that DB2 can exploit all that new hotness to achieve crazy fast performance, even though IBM (for whatever reason) chose not to exploit many of DB2's best features. Maybe one day someone will design a more relevant benchmark, in which the systems running it more closely resemble reality, but in the meantime, we at least have reports like this one that we can dig through for clues.

Photo pun courtesy of B. Shirley

New releases of DB2 software and DB2 magazine greet IOD attendees

DB2 Magazine: Volume 12, Issue 3

With Web 2.0 and mashups gaining traction as a compelling way to rapidly develop specialized database applications, it was only a matter of time before it hit the pages of DB2 Magazine. The cover story features IBM Web 2.0 expert (and PlanetDB2 blog buddy) Anant Jhingran, who describes how building a data services layer he calls Info 2.0 is essential to enabling Web 2.0 developers to build exciting services that all the cool kids will want to use.

DB2 Magazine's IDUG columnist, DB2 Gold Consultant Dave Buelke, writes about the environmental benefits of running just thirty z/OS mainframes instead of over 4,000 traditional servers. In the past couple years, this topic has resonated with me often because I keep running into database professionals who mention that their companies are moving to the mainframe mostly for environmental reasons. When you can pack that much computing power into a smaller building with less cooling equipment and only consume a fifth as much electricity, it becomes a strong selling point.

Here at IOD, Monday morning kicked off with a multi-part spectacle of noise and color, but many of us DB2 folks were just as impressed with Ambuj Goyal's announcement of DB2 9.5, which, after months of beta testing, is scheduled for GA release on Halloween. Its new thread-based architecture, sophisticated workload management controls embedded right in the engine, and malleable XML documents should result in a quick adoption of this powerful DBMS.

There's simply too much going on at IOD right now, but I hope to grab some time to describe more of it later this week.

Get the most out of next week's IOD conference

Lifehacker blog: Conference attendee tips

As you probably know, a conference is a different beast than a typical business trip, so I've collected some tips to help you better prepare for it. If you're not already familiar with Lifehacker, the award-winning productivity blog, their article on how to prepare for a conference is a good example of their ever-useful advice. Keep in mind, however, that their tip to explore the conference city in advance on Flickr may result in some NSFW images, since we are talking about Vegas after all.

The following tips come from my own sordid past, and from a committee I'm on that is planning next year's IDUG conference:

• Although people tend to think of a conference as several days of sitting in chairs, dodging flying bullet points, you'll also be walking a mile or two each day just to get around. After that, you'll probably be standing around for two to four hours a night at various receptions, unless your PHB is screaming at you to fight another production fire from your hotel room. Just in case you "accidentally" turn off your cell phone in order to mingle at the evening events, wear the most comfortable shoes you have in order to minimize pain and suffering.


• Bring a sturdy journal for note taking, and remember to grab a couple of decent pens from the office. Do not rely on the hotel notepads, which disintegrate quickly, or the free pens, which often fail.


• While in the exhibitors' area, put yourself on a swag diet. Unless your children are very easily amused, the odds are that nobody back home will want or appreciate the T-shirts or various plastic debris you collected from exhibitors. If you don't want vendors cold-calling you at your office for the next few months, don't take any of their swag. Don't get me wrong, a little bit of stuff is OK, just limit yourself to small sackful that can be easily carried onto the plane. For those of you suffering from an OCD that compels you to amass the largest heap of giveaways, don't depart for the conference without your company's FedEx or UPS number.


• Regardless of the temperature outside, it's a safe bet that the rooms at the conference will be kept slightly colder than a meat locker. If you have to wear a sweater in your computer room back home, you'll probably need one for the conference as well. In fact, just bring whatever you normally wear in the computer room, not only to show it off at the conference, but also to keep your jealous co-workers from wearing it while you're away.


• While exchanging business cards with new acquaintances (you do plan to meet new people, right?), take a moment to write a quick note about the person on the back of the card they hand you, even if they're still standing there. They'll most likely be flattered that you are taking the time to remember them for later, and you won't be racking your brain later on as you attempt to remember something about that guy or gal you met in some booth after downing your fourth Heineken. Just be discreet while you're jotting down a quick note, and, if possible, try not to let your new colleague see what you're writing about them.

Other than that, eat wisely, take it easy on the caffeine, and drink plenty of water. Call your family each evening (I've found that the break between the day's last session and the beginning of the exhibitor cocktail hour is a very good time), and don't fall behind on sleep. If I've missed anything, feel free to post a comment. See you in Vegas...

Boss photo courtesy of Matthew Lehman

Grab a sneak peek at next week's IOD2007 slides

IBM Information On Demand 2007 website: Download Conference Presentations

With IOD 2007 just days away, you've either reserved your seat in all the sessions you plan to attend, or you're the gambling type who's counting on a week of lucky breaks to get yourself into some very crowded rooms. No matter which path you've chosen, registered attendees now have access to advance copies of the slides for many IOD sessions. Just sign on to this site and you can start evaluating your choices. The downside is that the PDF files don't contain any of the speaker's typed notes, so they're not a very good substitute for attending the sessions in person.

By getting a head start on these sessions, you'll be saving yourself from the chore of looking through the conference DVD the night of the opening reception, resulting in more time for drinking. You're welcome!