Hak5 Beowulf Cluster

[TuxOtaku]

Hey guys,

After watching 406 and hearing Darren's idea for a Beowulf cluster, I know we all kinda got into discussions on IRC on what OS to use, the hardware best suited for the job, etc. So I figured a forum topic would be prudent.

So, in your experience, what OS would you choose for the cluster? what hardware would you use? Personally, after hearing ketin extolling the strengths of Plan9 as a clustering OS. I think that would be a perfect choice for the OS, what do you all think?

[devilish]

I've found this nice project on the interwebs the other day:

Link

Sounds interesting :-)

[psydT0ne]

That would be a sweet project, that the guys could cover over a series of shows. I'd be keen to see it done with som cheap low spec machines or even second hand rigs laying around.

I can't see myself springing for a shitload of shiny new magic bits when in the long run I'm not going to do a lot with it.

It would be a good project though, without a doubt.

[CodeBlue]

Hey guys,

here is a project I found the other day looks like a nice compact Mini-Cluster http://www.mini-itx.com/projects/cluster/?p

[Mike Chelen]

Darren and Paul were talking about the time it takes to do a couple frequent tasks, such as encode video and compress files. What software that runs on a Beowulf cluster could be applied?

[dylanwinn]

I personally think its a great idea! The only question is what it would be used for ;)

The Cluster should run Folding or Rendering within MOSIX on Damn Small Linux on the Mini-ITX Configuration shown below:

Price per unit is exactly $251.91, including a power supply and Shipping.

[Spoofy]

I've wanted to make my own beowulf cluster for a very long time. But on a personal note, I've never had an application for it - the hak5 guys do obviously with the video encoding, so thats not a problem. For me folding@home was never enough incentive to forke out 1500 or so Euro on such a project.

I dont know what anyone else is thinking regarding the processors etc, but my usual thoughts were:

* calculate the best value per mhz (since FLOPS isnt an available metric, at least nowhere i could find) available, taking account multiple cores.

* weigh the value-per-mhz value against the amount of power the processor consumes; afaik the highest cost of ownership of supercomputers/clusters and the like is the electricity bill, not the initial outlay.

I think its important to find some real hard info on what the advantages are of multicore over single core architectures. They're not linear for sure (4 cores vs 1, is definitely not 4x the power, maybe 30pc if your lucky). Im basically proposing taking a very scientific approach to this...

I think the end result is finding the cheapest, best value processor with the highest core count... which at the moment is the Intel i7 or possibly Core 2 Quad if memory serves me.

I think although the mini-itx idea seems sound, your cost-to-performace ratio would be taking a massive hit in exchange for size. But it should take alot less power, so... I'm not sure. Its a complex issue that needs alot of discussion i think :P

[dylanwinn]

I think although the mini-itx idea seems sound, your cost-to-performace ratio would be taking a massive hit in exchange for size. But it should take alot less power, so... I'm not sure.

Hmm... ROUNDUP TIME!

(All estimates. Prices from newegg, except on micro-itx. They include mobo, cpu, hdd, psu & ram. No cases.)

Mini-ITX (Atom)

Mhz: 1.6 Ghz x 2 Cores = 3200

RAM: 512 MB

HDD: 128 MB SSD

Power: 30 Watts

Cost: $251.91

Ratio: 0.078721875

Micro-ATX (Q6600)

Mhz: 2.4 Ghz x 4 Cores = 9600

RAM: 4 GB

HDD: 80 GB SATA

Power: 100 Watts ?

Cost: $447.38 (DIRT CHEAP!)

Ratio: 0.04660208333

Full-ATX (Core i7)

Mhz: 3.2 Ghz x 4 Cores = 12800

RAM: 6 GB

HDD: 80 GB SATA

Power: 60 Watts ?

Cost: $1,594.98

Ratio: 0.1246078125

So, it looks like after all, the good old Q6600 comes out on top. With a price/performance ratio of ~ 5 Cents/mhz, it beats the Core i7's ~ 12 cents/mhz, and even the Atom's ~ 7 cents/mhz! Guess you were right!

[cooper]

I've been thinking a LOT about this particular subject, but decided to shelve the idea some time ago due to time and money constraints. The way I would do this would be to have 1 big disk attached (or 2 in raid 1... whatever floats your storage boat) to the master node which would act as an NFS File Server to the other nodes in the cluster. Linux, when given a network adapter with EEPROM, can be told to boot off of an NFS fileserver, which would allow the other machines in the cluster to be diskless, saving cost as well as power and allowing cleaner wiring.

Note that a Jetway Atom board on NewEgg cost just $99 and if you allow $13 for 1 gig of value ram that box costs a mere $112 which comes to ~3.5 in the totally meaningless Cents/MHz metric.

No post so far mentioned the cost of the power supply and the chassis, which I'll assume to be $100 and I'll also assume in the next calculation that these parts will have the same economic lifetime as the rest of the box. As the boxes will not carry any HD's, I'll deduct $35 from the listed cost for that.

Now, assuming you're using a node at 100% load non-stop for 3 years (well, allow a bit of downtime to make the calculation easy. 1000 days rather than 1094) and then replace it, and electricity costs $0.1052/KWh (source) the total cost of each node comes to:

Atom: $212 + 30*24*0.1052 = $212 + $75.75 = $287.75 = $95.92 / year

Q6600: $413 + 125*24*0.1052 = $413 + $315.60 = $728.60 = $242.87 / year

i7: $1560 + 180*24*0.1052 = $1560 + $454.46 = $2014.46 = $671.49 / year

The Q6600 eats 105 at the CPU, and allowing some power use by the mobo I'm assuming the machine as a whole to consume 125W, which is a LOW estimate.

The i7 eats 160 at the CPU so I'm assuming it to be a 180W box with the same LOW estimate as used in the Q6600. Remember that for each additional W that the machine eats you pay 1*24*0.1052 = $2.50 over its economic lifetime.

Another thing about Watts... All that power that the machine eats comes out in the form of heat. When you put 6 nodes eating 125W each into a room that 750W output will raise the temperature by a few degrees at the least, and is likely to require more work from your airco unit which will again raise cost. Oh, and you'll probably want to make sure the wiring and fuses in your house leading up to this cluster can take that strain. If it can't, time to whip out the wallet once more.

If each node takes just 30W the total for the entire cluster is a mere 180W which shouldn't be noticed that much.

[sjbwiii]

Started on my cluster - got the idea from the mini-itx one.

Won't be the fastest thing - more to learn / play with.

Will be a 10 way - more if i can get the boards - cost $10 each.

http://s109.photobucket.com/albums/n76/tur...rrent=comp3.jpg

[psydT0ne]

here's a funky idea...and i have no idea if this is even remotely possible...

our fons right...expanded with sd cards..

is it possible that there may be some kind of small linux os floating about out there that could be installed onto a fon...and therefore by networking a number of fons together via a switch (ethernet) or even wireless that this cluster could be achieved???

i mean it might not be very powerful...but could it be achieved..????

thoughts??

[SomethingToChatWith]

taking a cluster of routers to brute-force some hashes or do other high-demand tasks? unless you're talking portable cluster here it might be but i think a bunch of computers would be a little faster...

[psydT0ne]

i didn't think it would be super powerful but yeah maybe something cool like wpa keys or some such.

just an idea tos all :)

[caninomeister]

Hey im thinking of trying to set up at least one computer lab in my school as a temporary beowulf cluster computer, ( my programming teacher hopefully wont mind, she likes students like me ;) ) and im thinking i will use chaos for the slave nodes and clusterknoppix for the master node. Any advice or suggestions would be great!

[stingwray]

Beowulf clusters are cool and all, but hardly anyone has firstly a use for all the power (at any time or all the time) and secondly decent enough applications for it to solve their problems.

If people are interested in playing with beowulfs and setting them up etc. Might I suggest using something like Amazon's web service. You can set the networks up, add machines, take the off again and play to your hearts content. You can also do something calculations and when your not using them you can turn them all off.

Obviously its not the best solution for everybody, but by the sounds of things from this thread it may suit one or two of you.

[metatron]

We had this same talk two or three years back. I've set up a few but I'm really not a massive fan of clusters, as I tend to like parallel processing for home use as distributing a large work load on cheap hardware is not great. As someone that has worked on computational fluid dynamics systems and other systems that do big maths, I have to say your better off just buying an 8 socket AMD motherboard and filling it with as much memory you can and a sata Ram Disk.

You could even build a system around FPGA's which would be very quick for some tasks.

[Gibbo]

You could even build a system around FPGA's which would be very quick for some tasks.

Seconded, I've previous experience with using DSP's for performing processes and the results outweighed using clusters, not to mention the cost. I guess it all comes down to the specific task at hand.

:o

[aloishis89]

I built my own cluster last year using Cluster Knoppix. It has I think 9 nodes right now with several more on the way. Because my budget was $0, I used any computer I could get. This ranged from a crappy old box with like a 900MHz proc in it all the way up to a server with dual 2.4GHz intel Xeons and 4 gigs of ram. Of course, the balance of this thing is not great at all, as the more powerful machines take most of the load, but the main purpose of the project was to get the thing running and distributing the load across all the machines. It is all very experimental. I know there were some people here wanting to build their own. I suggest you go for it, its really cool and I learned a ton.

Here is what it looked like while I was still working on it (before it was tidy):

That picture always makes me laugh.

[gcninja]

that looks more like a cluster fuck than anything... what do you use it for?

[aloishis89]

In the past, I had it doing Folding@home for a little while and I planned on using it for hash cracking. It isn't running right now because I have no place for it to live. After I first built it, I left it on 24/7 in my room and when I would wake up in the morning, the room was at least 10 degrees hotter than when I went to bed. Plus having that many fans running gets kinda loud. Anyway, I mostly use it for experimenting with clustering, networking, and pen testing.

[stutte]

hello i'm new to the forum .

i'm trying to make a playstation 3 cluster +2 dual core's + dual core laptop

i already have fedora running on the ps3 ... can acces the pc's over the network .

but i need a tool to set up the nodes and the server ?

i think the cell broadband engine in the ps3 can do some serieus number crunching ... ;)

greetz from europe

[Resix]

Ok so I read that people don't know what to with their clusters.

Well maybe there are people that do have decent applications, but don't have a cluster to run them on.

How about making some of your clusters accessible in some way, request system or something, to have people use your cluster?

Just a thought...

[AltarCrystal]

I actually do have some experience in programming for parallel processing (actually took a really interesting class on it last semester), however, I can't really justify the need for buying hardware to build one. I have no current application, nor do I have one in mind I want to write.

I did, however, create what I liked to call a "PicoCluster". 6 laptops networked together running PelicanHPC. This was actually something I whipped up because I had a big project to work on for the Parallel Processing class I was taking, but had to be in a city 4 hours away over the weekend. With it, I developed a Google Maps-esque application.

But really, why dump all the money into something that will jut be running something like F@H?

[Cthobs]

I've had the itch for building an XBox based Beowulf cluster since the original XBox came out. I bought one off of eBay last month and have finally gotten it cracked. Going through the xbox-linux.org distros now to see which one I want to work with.

For $200 you could have 4 XBoxes running.

I also want to look into the current interest of using GPUs as standard processors. Without having the XBoxes connected to video, it's additional cycles sitting idle.

As soon as I get the one set up and configured, I'll order the second one and get it clustering.

[stutte]

i know there is a port for the aircrack-ng on source forge that runs on ps3 . dont know if its fast or not .

the fastest supercomputer contains the cell chip from ps3

linux runs great on the ps3 pretty simple

you could make it do live dictionary attacks

if i could only program in the cell sdk

nice topic .

trust your technolust rules