Update on the XML Challenge

December 3, 2008

I just found out that more than 55,000 people have entered the XML challenge so far. Remember that some contests in the challenge are run on a monthly basis, so there is still an opportunity to win prizes. And I understand that there are still more than $50,000 in prizes that have yet to be won in US and Canada. I also heard that the organizers still have t-shirts to give away for people who simply enter the contest in the US. But these are running out fast, so don’t waste any time with your entry.

The two programming-oriented contests in the challenge actually opened on the 1st of December in the US (there are separate contests in different countries). Entries can be sent in until the 31st of January. Good luck with your entries…


6 Responses to “Update on the XML Challenge”

  1. Dave Horsman Says:

    Hi Conor,

    Um. No disrespect intended in any way, your previous introduction of a topic being a good way to get some discussion rolling…. but uh…. it’s kind of your turn to make insightful suggestion or commonent on the use of XML. How many technical writers are there at IBM anyway?

    We had some excellent feed back on quantity of XML nodes in a few domains and general expectations of complexity so it’s your turn guy. Toss a topic out there. What’s the lab doing these days?


    Dave H.

  2. Dave Horsman Says:

    Excuse my editing. I hope you spell check these posts… Dave H.

  3. Hi Dave,

    I appreciate your desire for more posts on this topic. I have not had an opportunity to write many posts recently. Like many people, I am quite busy these days, working nights and weekends. This blog is a personal blog, where I share information that I think may be of interest to people in and around this field. If it is helpful in any way to people, then I am happy. Of course, some of this information does have a decidedly IBM flavor, but that’s because that is what I am primarly exposed to. I see first hand organizations evaluating their options, organizations deploying this technology, and I see the results that organizations then enjoy. When I come across an intertesting topic, I share the information with readers of this blog. But this blog is a personal endeavor. And because it is a personal endeavor, I can contribute only when I have the time to do so. I’ll get to writing up a few posts soon…


  4. Dave Horsman Says:

    9 Dec 08

    Hi Conor,

    Thank you for your kind response, I don’t doubt you are a very busy guy. The additional clarification definitely helped and I commend your efforts and more than adequate contribution of your expertise. Regarding an IBM slant that is quite appropriate and what I think most folks are looking to get from this blog.

    Regarding IBM, we all have our respective roles to play in a massive industry and I perceive IBM’s primary focus on providing work-horse high capacity solutions a necessary field well handled with insight and customer focus. Big Blue’s efforts in research being a very important if secondary effect.

    I like your blog but I get the impression I may be a bit lost in terms of links, thinking this was the home link re the XML challenge. If I may comment respectfully on the challenge in a general way I would say this.

    A programming challenge re XML is of course important. However I feel the true challenge that exists with XML is from a chronological and conceptual stand point, the contextual schema’s themselves Creating these huge schemas (that are in fact condensed formal definitions,) is a mammoth task often provided free of charge by many industry experts in both IT and specific industry domains. The formalized grammar involved requires careful thought, collaboration and compromise and I feel these folks should always get first mention and our appreciation for their efforts.

    Though a developer, a code challenge takes second place in my view. Having said that this in no way detracts from your contest nor goal in this regard. I view IBM’s domain historically has been mainly large scale high volume transaction and commercial processing and this continues and as such I would expect much of the efforts to centre on Native XML, discussion on hybrid blends, shredding, performance achievements and strategies, DBA issues, hardware and the like. Again several of many important contributions needing sustained effort.

    Having said all of that. I have yet to notice the real coding challenge of XML to be well and truly started. What is required is complete rethinking of coding and development that is centred on these schema and the cognitive and linguistic science driving it all.

    XML may have started off as a historic effort at standardization, normalization of data and documents, and an overall effort to improve service to clients. However, like the almost hidden revolution of object oriented technologies behind what was to the layman perceived as graphical user interfaces and desktops, XML has some much deeper implications than the important goals already mentioned.

    In many respects I feel that much of our discussion in IBM challenge is somewhat pedestrian though difficult. To be fair, most technological advance is layered on top of pre-existing foundations and in this fashion, there is a need to “role” XML into IBM existing database technologies. This sort of process is a topic in itself.

    In that vein, in writing preliminary drafts for your “contest”, I have found myself primarily discussing and summarizing the existing state of SQL and object database technologies with a particular focus on IBM.

    I find this in some ways ironic. For like most technological advance, the original ?discovery? in these two technologies then became quite static with endless debates going on amongst various purists, advocates and camps. Like other scientific communities, these debates have often had aspects that were heated, emotional, pseudo-religious, political, monopolistic… I’m sure you get the point. In part, all discovery follows an application phase where it essential pays for itself, idea mature, become stale and radically new innovation ensues. Socio-economic sciences are not really my domain of expertise anyway. The point being that in kind, we have seen minor advances, insights and achievements but not much real advance. I would say this applies equally to DW and BI, as in oh wow, a dashboard. That should get a response.

    Regardless of the irony, I must again emphasise a personal view of the need to discuss your existing database technologies in some depth before really getting into the meat of your challenge. So I well understand your previous post bringing RI into an XML forum.

    I felt these discussion might come across as off topic or tiresome when appearing in your area of pages but I am not only a guy who likes a challenge, but quite willing to take risks and expose myself to criticism by presenting unfashionable concepts and opinions. Though it might trigger more endless repetitive debate I see no avoiding it.

    I guess because of the manner in which the promotional material crossed my virtual desktop I have gotten the impression that the IBM XML Challenge is your baby Conor and your blog an ideal location to hash it out. I’m not quite sure if all of this is asking for your blessing in this regard or correction on where these posts should be vetted. Anyway, time allowing, what do you and the other readers think of all this?


    Dave H.

  5. Hi Dave,

    I apologize for taking so long to respond. It is a very busy time of the year for me right now.

    I am not directly involved in the XML challenge. However, I do know some folks who are involved. And while this incarnation of the XML challenge is in full swing right now, I know that they are currently working on planning next year’s set of contests. I will certainly pass along your input to them. I think you make some very good and valid points.

    Creating the schemas is a complex challenge. And credit should indeed be bestowed upon the individuals who contribute to these efforts, as well as the organizations who in effect sponsor their contributions. Their efforts are helping raise the levels of standardization and information exchange in many very important industries.

    You are correct. There are larger implications for XML. And there are a number of phases of “evolution” that the industry will undertake over time. The challenge for those of us who want to see XML fulfill its potential, is to synchronize our efforts and messages with the state of the industry. That way, we can help the industry get to the next stage at each step in the evolution.

    For instance, the reality is that not enough people today understand the benefits of XML databases. I see it all the time when I speak at conferences. When storing XML, the majority of people are still using shredding and stuffing (aka LOB storage) into relational databases. These are techniques that began as workarounds for storing XML in a relational database. They are perfectly valid in some instances. However, in many cases, they are not the best approach.

    (Its easy to get caught up in the leading-edge. But we must realize that we need the majority of the market to get on-board to make real progress in the industry.)

    So, the next step is to have the majority of the industry use native XML storage. When they do this, they will see that some logic has moved from the application layer to the database layer, and they will encounter a number of benefits as a result. Then they will be ready for the next step in the evolution.

    What’s that next step? That’s the big question. In my opinion, that next step for environments where analysis is important, is a set of breakthroughs that allow for the robust analysis of information in XML format. This is a real challenge with today’s infrastructure.

    A more general step in the evolution is… a set of integrated tools across the different roles that touch XML data: data architects, application architects, database architects, database administrators, application developers, and so on. At present, organizations have many challenges when dealing with the disparate tools (or lack of adequate tools) across these roles. I think that addressing this issue is the next major stage in the evolution. I would love to see integrated data management tools become as commonplace as integrated software development environments.

    The idea behind the XML challenge is to help people realize how easy it is to work with native XML storage. We are hoping to help people realize that native XML storage is easy, and that it is the best approach for certain use cases. And we are hoping that can help get people to the next stage…

    By the way, there is a perception that IBM caters to the high-end of the market when it comes to performance, scalability, reliability, and so on. This is true. However, IBM also makes these very same capabilities available for no charge. For instance, the no-charge version of DB2 (which is called DB2 Express-C) has the very same native XML storage capabilities as DB2 Enterprise. There are no limits to the amount of data you can process with DB2 Express-C. The only limits are regarding the processing power and memory in the server. We are finding that many small companies are embedding DB2 Express-C in their own products because they get leading native XML capabilities, there is no charge, and they get great performance and reliability.

  6. Update: I see that, just before the holidays, the organizers issued a press release about the XML challenge and the number of entrants is now up to 70,000.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: