Jump to content

Distributed compiler


macker

Recommended Posts

I want to install Gentoo on a load (27) of old laptops and desktops (mostly P2’s to P4’s), my idea is to build a version of SLAX (GNU C/C++ compiler, GNU Make, distcc, ccache, dmucs, ccontrol, crosstool, distccWebView and tcpbalance) and burn a load of copies, then have them run on a large number of universities computers, so I can cut the Gentoo install time, by using a distributed compiler.

My question is, are there any live discs available that can do this, or is there a better way to slash the install times?

Link to comment
Share on other sites

Well, I would think that if the install time mattered that much, you would simply install Gentoo from precompiled packaged.

NOTE: The following is written without any idea how that toolchain works in detail, though I grasp the concept.

Once installed you can network the toolchain up so you have a full network at your disposal as a compilation farm of sorts. Presumably you could then just work on compiling system-optimized versions of the programs in question.

What worries me is the simple fact that those systems are far from homogenous, and I don't know if distcc will allow you to... cross-optimize (?) for the machine the binary is to run on eventually.

If this isn't for the fun and/or learning experience of it (which I'm sure would be most satisfying) I'd not bother getting the very last bit of performance out of the systems. I'd go generic and be done quickly.

Link to comment
Share on other sites

Yeah, it's more for, fun and/or learning experience. I’ve got a stack of old laptops and desktops I picked up from local companies (dumpster diving), and I have some free time this weekend, so I thought it would be interesting to play around with distributed compiling.

Link to comment
Share on other sites

I'm planning a heterogenous cluster (10-12 nodes) using Mini-ITXes. Just for the fun of playing with a cluster. Got the centralised filesystem going already, so this distributed compilation thing should be fairly straightforward once I get to that.

Link to comment
Share on other sites

I’m living in student accommodation, so I’m limited on space to build a cluster. I help my universities IT team out from time to time (mainly coding C/C++), in exchange for access to one of the computer rooms out of hours, so I have access to around 120 computers and a server room.

It’s going to be a few years before I can find the cash for multiple Mini-ITXes, but it sound like a cool project.

Link to comment
Share on other sites

That's really cool, but given that, wouldn't you lose a lot of time transferring the source files to the machines?

Also, 120 machines... I doubt you'll need that many with anything other than Open Office.

Link to comment
Share on other sites

You would lose a chunk of time but it’s more about playing around with distributed compilation then time, although one of the reasons I’m trying it, is to see what kind of time will be involved, and if there are any big issues that arise when spreading the task over a large network.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...