About

The Systems Biology Kbase pilot hardware named kandinsky is a new machine designed for optimal support of the Hadoop architecture/runtime. The original recommendation made by the science advisory board focused on examining the MapReduce programming paradigm and it’s applicability to bioinformatics applications. As a result, the hardware configuration includes over 0.5 petabytes of storage on local nodes under the direction of the Hadoop File System.

In addition to supporting Hadoop based applications, support for virtualization will be added. As experience accumulates, an understanding of the compatibility of a system that supports virtualization and the execution of remotely created images and the Hadoop runtime will become more clear.

The system was purchased under the Knowledgebase R&D project sponsored by the Office of Biological and Environmental Research in the DOE Office of Science with American Recovery and Reinvestment Act 2009 funding and performed at Oak Ridge National Laboratory (ORNL). ORNL is managed by UT-Battelle, LLC, for the U.S. Department of Energy under contract DE-AC05-00OR22725.

One Response to About

  1. brettint says:

    Hi Brian,

    I’ll take a stab at answering your question. The short answer is we anticipate use of virtualization to support running various images. The devil is in the details, though, due to different virtualization technologies. More to come on this. Can you tell us more about your images, hyperviser, etc?

    The longer answer is the system specification was based on the needs of a hadoop cluster. This was in the original recommendation that was reviewed by a team of advisors. So there is a lot of storage on the local nodes. There is also shared network attached storage. We also see how virtualization can co-exist on the machine. Finally, your typical cluster like jobs, torque-sge-whatever, will receive the least of our attention.

    Tom

    —–Original Message—–
    From: Brian C. Thomas [mailto:bcthomas@berkeley.edu]
    Sent: Wednesday, June 23, 2010 11:21 AM
    To: Cottingham, Robert W.
    Cc: Jill Banfield; Pan, Chongle; Brettin, Thomas S.; Kewalramani, Avinash R.; Galloway, Michael D.
    Subject: Re: Kbase computer resource

    Hi Bob,

    Looks like a substantial collection of hardware. I have a quick question for you which I don’t find an answer to anywhere on the blog: Is this computer resource going to be configured as a cloud, i.e. configured to support scalable numbers of virtual servers running various OS-images, or is it planned only to be used as a compute cluster, i.e. a single, queue-based platform for submitting jobs? The reason I ask is that I have already created numerous virtual machine images customized for various tasks and it would be convenient to be able to drop them on a new resource and run, as opposed to setting up the environment for our processing tasks in a queue-based system (which we already have locally, but not near the magnitude of your compute cluster).

    Thanks,

    bct

    Brian C. Thomas, Ph.D.
    Metainformatics/Bioinformatics
    University of California, Berkeley
    Berkeley, CA 94720 USA

    bcthomas@berkeley.edu

    On Jun 22, 2010, at 8:50 PM, Cottingham, Robert W. wrote:

    Dear Jill,

    You can start by visiting the blog http://sbkbase.wordpress.com/ which may answer your questions but if not please post questions there or email Tom Brettin, Avinash Kewalramani and Michael Galloway for information (cc’d above).

    We are in the midst of making the necessary security changes so users external to ORNL may login. This is what ORNL refers to as the XCAMS system and more info can be found on the blog however it will be a few more days until this is completely set up. In the mean time please engage us in a fuller dialog to understand how your project can be developed and piloted.

    Look forward to working with you and appreciate your endeavors to pioneer Kbase.

    All the best,
    Bob

    We could very much use access to computer resource you mentioned for our Kbase project. I understand that you agreed to give us an account.

    The first thing that we’d like to request is some information about exactly what the computer consists of and how to arrange an account. What services are currently available?

    This is very exciting. Thank you for agreeing to share the resource with our (and other) groups.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s