help me with my vmware file server plans

[ace_ventura]

what i want to do is build a computer that will host multiple virtual servers so im looking at getting a quad core cpu with 8GB of ram, i would also like this box to have lots of disk space for the VMs that is also redundant so i have a lesser chance of loosing data, so for this im thinking 4x 750GB drives in a RAID 5.

Now what i have planned so far is to have server 2003 be the host OS for the box, then install VMware server and create away. One of these virtual servers will be a file server that i would like to be fast, i would like to see at least 100 MB/s transfer rates in the vista file copy dialog box, now i know thats not accurate or a great measurement but if its going to read that fast it should be plenty fast in actuality as well, and yes my network is gbit and yes i do know it will depend on the speed of the other computer transferring the file to and from, i just want the server to be capable of 100 MB/s of faster (my desktop has a raptor raid 0 array so that should hit 100 MB/s anyways)

My plan is ~500GB for VM space and the rest ~1.7TB or so to File server space (4x750 RAID 5= ~2.2TB), so i would have 2 partitions: 1 500GB and the other the rest of the array so ~1.7TB. So the easy part is i will create for example 4 VM boot drives on the 500 GB partition now one of those VMs will be the file server.

The dilemma is how i create the FS data drive, because i dont want to create a virtual drive that will expand automatically since that will lower performance and there could be a big ass 1TB file on this partition and i dont know how well that would play with vmware and or windows, and it would take a really long time to create a fixed size large 1.5TB virtual drive to mount as the 4th Virtual FS data drive, plus if i needed to expand it for more space it would take a very long time plus there's a greater chance of data loss if the virtual drive becomes corrupt in the midst of expanding the virtual hard disk so that leaves this other option that i have come across which is to use a physical disk in vmware which sounds like what i would need however i would think that the large left over partition ~1.7TB should be unmounted on the host os, in my case server 2003, that i will use for this pc because i wouldnt think it would work very well to have 2 OS's having access to 1 partition at the same time, so ok, now if i were to use a physical disk using the left over partition and i unmounted that partition in the host os and all is well i can store files on it using the virtual file server, but if i were to shut down all VMs and remount the partition i would be using for the virtual FS would i see the data on the disk? i dont think i would if i can remember correctly from a previous test which can make it more difficult to get access to data should something happen therefore i would also want to have a NAS for backup and setup a backup program on the host os that will backup the boot VM's and setup a backup app on the FS to backup all the data on the FS data partition all to a NAS of equal size to my 2.2TB raid array just for cushion. But heres another scenario, what if i want to resize the partition on my raid array so i can have more space, so heres what i would do; boot to G-Parted and resize the partition so i may have 250GB for VMs and then 1.95TB for FS data, now when i restart the vms i dont think vmware is going to like that, so i could loose data.

so i guess whats the best way to set this all up for speed and data redundancy, if i got another computer and built a SAN out of it, that may work, but thats extra cost that im trying to keep low

[Sparda]

You should fit as many network cards as there are spare PCI slots. Ideally binding each virtual server to it's own network adapter. Bandwidth++

[VaKo]

There are a few things I would do differently. Firstly, I would fit a 120GB disc in addition to the RAID array, and use this to install the OS and applications on (I would configure a Poweredge 2950 or similar with 2 disks in RAID1 for the OS and applications and 4 discs in RAID5 for the data, and this works well for my Hyper-V test lab machine). I would also configure the host OS as the file server, so you avoid the overhead of a virtual machine. It will also make managing disk space etc far more easy than using a virtual machine for a file server. (There is an option in VMware that allows a virtual machine to use a physical hard disk instead of a vhd file but I suspect this won't work so well with a RAID array, unless your controller card allows you to configure slices of the array as seperate logical disks). The you can run your virtual machines in addition to the fileserver, rather than attempting to shoehorn a production NAS setup into a VM and wasting time fighting the inadaquacies of virtualization.

This is how we are planning on configuring some Branch Domain Controllers for our remote offices at my workplace, a 2950 running Windows 2008 + the Hyper-V and File Server roles, the physical machine serving as a file server/local DFS cache and a 2nd virtualized install of Server 2008 running as a BDC on top of this. A similar setup might suit you, although I could only recommend Hyper-V if you are purely doing windows virtualization.

Sparda's solution of adding Nics will help, I would even recommend looking a link teaming, using 2 NICs for your fileserver interface, and adding extra NIC's for your VM's to share. Link teaming won't make it faster, but it will deffinately help with multiple clients trying to access the file server concurrently. It would require that you adjust your network to support this though. I would also recommend ditching 2003 and replacing it with Server 2008 x64. It is a world better than 2003, and is ridiculusly easy to setup & use plus it comes with a huge variety of roles and features.

[ace_ventura]

what i was planning for virtual machines were a pdc (primary domain controller srv08), fs, www and vpn server and i was planning on building this myself but if its cheaper with your option i may do that instead

so if this is the case, i would want to have my file server part of the domain which would be running as a vm, so i dont know how well that would work.

actually i do have a few spare older computers if you were to suggest using a bdc like what you have and just run a pdc on an older box

[VaKo]

I would put your PDC on a seperate machine to be honest. Use your uber machine as a 2008 file server, split your raid into the 500GB for vm's and the 1.7TB FS partition (setup a domain based DFS while your at it), then just setup a vm for a BDC (we're looking at 2008 core RODC's here) and a seperate vm for the WWW and VPN stuff. Much more easy to maintain like this.

[taiyed14]

You might want to look into using Linux as the host OS. This would cut down on the amount of resources being used by the host, and allow for more resources to be available for the guest OSes. VMware Server runs nicely on Linux.

If that isn't an option, then I would agree with VaKo. Make the host OS the FS and put an extra HDD in the machine that will be used for OS and applications, this way, if you need to blow away the OS for some reason, you don't need to worry about the data being erased. This will also give you maximum storage space on your RAID.

[VaKo]

I wouldn't recommend linux in this case, simply because you have to mess about with Samba and 2008 domains if your planning on using the same box as a file server and vm server. If you want a lite OS for the vm/fs server and your going to use it in a domain then you should look at Server 2008 Core and hyperV.

[ace_ventura]

i looked into and thought about doing Hyper-V but i know vmware and use it at work so thats why i was leaning vmware

[VaKo]

I'll admit that the interface for VMware is a lot more mature but hyper-v is a good product.

[ace_ventura]

so far i gather this is how i should set this up:

Old PC: PDC

New Server Build:

2x250 RAID1--For Host OS: 2008x64

4x750 RAID5--For VMs and FS Data

Quad Core

8Gb RAM

----------------------

Host OS 2008x64 as FS w/data on RAID5

software:

VMware Server

BDC VM on RAID5

WWW VM on RAID5

VPN VM on RAID5

PCI raid card for RAID5?

mobo RAID1 for Host OS?

Other Hardware:

2TB NAS esata prefered or USB (for backing up RAID1 and RAID5)

all of the OS's will be joined to the PDC's Domain

this is for a small network so i dont think i will need the extra nics, the mobo will have 2 nics built in but if its better let me know and ill throw in however many PCI NICs i should get

[VaKo]

What you have put in your plan is pretty much how I would do it, but if someone else has a better idea then do that. The hardware setup looks damn good imo, use the mobo for raid 1 and you will need a separate pci-e card for RAID5. The nics aren't really an issue unless you are going to have lots of traffic going in and out your VM's or are using link teaming. For this setup your vm's will be happy with sharing a single link leaving the other link for the file server.

[ace_ventura]

the only thing thats a downer but i guess not to big of a deal, is that if i need to apply patches to the FS everything else must go down, thats why i originally had thought to do a FS in a VM, but making the host OS the FS does seem to alleviate the issues i may have with a virtual disk as described above in a few scenarios

[VaKo]

Is that a problem if it happens at 4am on a Thursday?

[ssmithisme]

I would also recommend that you use the host OS as the file server because of data integrity issues.

Thats a pretty nice system you want to build too, You should put Windows Server 2008 on it.

I think you should have a separate disk for the OS and applications to be installed on, and make the big RAID dedicated to storing stuff.

[ace_ventura]

Is that a problem if it happens at 4am on a Thursday?

nope thats not an issue :)

[ace_ventura]

I would also recommend that you use the host OS as the file server because of data integrity issues.

Thats a pretty nice system you want to build too, You should put Windows Server 2008 on it.

I think you should have a separate disk for the OS and applications to be installed on, and make the big RAID dedicated to storing stuff.

i plan to put 2008x64 on it, and i will do the separate disk thing as well

[adam0101]

ESX i is what you need. Its free and pretty easy to setup, much more mature then microsoft's offerings and the footprint is tiny.

http://www.vmware.com/pdf/vi3_35/esx_3/r35...ation_guide.pdf

[DingleBerries]

ESX i is what you need. Its free and pretty easy to setup, much more mature then microsoft's offerings and the footprint is tiny.

http://www.vmware.com/pdf/vi3_35/esx_3/r35...ation_guide.pdf

Ive also heard that ESXi was really good.. havent had a chance to play with it, but from everything ive read it sounds ideal for this situation.

[VaKo]

ESXi does indeed look good (thanks for the link, will be demoing this at work), but in this case using it means you have to virtualize your file server.

[ace_ventura]

ESXi does indeed look good (thanks for the link, will be demoing this at work), but in this case using it means you have to virtualize your file server.

in which case if i virtualized my fs then i would run into the disk issues situation described earlier

[Rain]

This is a tough ground.

The idea of having everything VM is great. But VM's are unreliable. they have the standard of crashing and no data recovery if the disk was being written to. in my experience the VHD sometimes has some problems expanding to unless you want to allocate all disk space ahead of time but that can take HOURS!

So best bet it to have only things like VPN and telnet/vnc servers in VM. FS's would be great because they don't require much CPU and overhead unless it is used 24/7. But again it scares me to wonder if all my datais slwoing be corrupted. ** kinda one of those things i lay awake at night thinking about.**

But ESXi is GREAT!

i habet had a problem with it yet.

** i still don't trust MY data to it though**

Let us know how it works out!

NAS and SANS - that is just me tho i am crazy about Data and recovery times :(