anyone manage/use a hadoop cluster?

Discussion in 'Business & Enterprise Computing' started by username_taken, Mar 12, 2012.

  1. username_taken

    username_taken Member

    Joined:
    Oct 19, 2004
    Messages:
    1,352
    Location:
    Austin, TX
    Building a hadoop cluster for some stuff at work (datawarehouse type stuff) and I've been amazed at how powerful it is and how much really interesting stuff people have done with it, and the sheer size of some clusters out there.

    However it's one of the steepest learning curve things I've come across, not from a sysadmin point of view ( especially with cloudera CHD stuff avaiable ) but from a user point of view. Coming from a non-developer background its quite intimidating.

    Just curious if people out there in OCAU world are using it.
     
  2. Jase

    Jase Member

    Joined:
    Jun 28, 2001
    Messages:
    196
    Location:
    Sydney 2081
    No, but I heard of a customer spending a lot of money on infrastructure to build hadoop and then finding out they can't "plug it in" to anything which they assumed they could because they thought it was "big data".
     
  3. elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    44,218
    Location:
    Brisbane
    We're in the process of building a hadoop cluster for a Pentaho BI system we're investigating.

    We've got a particular data set that's 11TB in total across separate databases (although they share keys that link them together) that's proving far too difficult to query by normal methods. The hope is that with some hadoop/mapreduce magic and Pentaho BI with Kettle (their ETL tool) on top, we can build a fairly easy report template system to keep our marketing and exec teams happy.
     

Share This Page

Advertisement: