I am
currently running home lab for learning virtualisation and new technologies.
This system was integrated with GA-B75M-D3H motherboard, i7 -3770 @ 3.4 GHz by me.
I was
craving to add an additional system in my lab with an Intel Haswell CPU from the day this Processors were available to public. However I wanted
to confirm about any known issues before burning a hole in my wallet.
These new
Haswell motherboards are either installed with Realtek or Intel based LAN
controllers and new SATA chipsets are provided by Intel B85/H87 or Z87
controllers. These on-board controllers may impact the ESXi installation or give serious PSOD errors or booting problems which could put the entire build in vain.
Famous bloggers vladan
and Ivo Beerens built their system with Asrock Z87
and GA-Z87-D3HP motherboards respectively. They have mentioned about the on-board
Gigabit NIC I217V is not recognised by ESXi and had to go with add-on NIC.
I decided
to try with Realtek chipset based motherboard which may be supported by ESXi
native drivers as it worked without any issue in my previous B75 based
IvyBridge system.
I bought a GA-Z87M-D3H Gigabyte motherboard and Intel i7 4770
CPU. This motherboard supports 4 x 8 GB DDR3, 6 x SATA3 6 Gb/s ports, 2 x PCI
Slots, 1 x PCIEX16, 1 x PCIEX4 in Micro ATX real-estate.
The
reason I went ahead with this motherboard is based on below assessments:
- Micro ATX form Factor - will keep the system footprint smaller
- Entry level Z87 Chipset with reasonable price - This system is not for gaming purpose
- Supports up to 32 GB DDR3 memory - can run more VMs
- Supports VT-d - To test device passthrough
- PCI 2.0 - In case on-board NIC backfire
- 6 x SATA3 6 Gb/s ports (Can be converted as NAS system with i3 or Pentium processors)
My
Haswell based test box consists of following parts:
- Processor: Intel
i7-4770 @ 3.9 GHz with 8 MB Cache x 1
- Motherboard: Gigabyte GA-Z87M-D3H x 1
- Memory: G.Skill Ripjaws-X - 8 GB x 3
- Network Interface cards: Intel PRO/1000 GT x 2
- Computer Case: CoolerMaster RC344 x 1
- Boot Device: SanDisk Cruzer
Blade - 8GB
Installation
and Testing:
Installed
ESXi 5.1 trial version in the USB Drive was smooth. Unfortunately Realtek On-board
NIC is not supported by ESXi. So I had to use the PCI Intel NICs that I bought
from e-bay. These add-on cards are detected by ESXi without any issue. I have
configured and tested these cards with 8192 Jumbo frames as well. I am really
happy that I bought couple of these cards in spare which came as handy during
this build and the price of the network cards are very much cheaper. On-board
SATA controller was detected by ESXi. I tested this by connecting a SATA drive
which I pulled out from another desktop.
Before
buying this motherboard I read the manual and confirmed myself that VT-d is
supported. So my next "test item" was to confirm about VT-d support
which will allow us to passthrough the devices to test VMs. I logged in through
vSphere Client and selected configuration -> Advanced settings ->
Configure Passthrough option.
Only
below given controllers are visible in Z87M-D3H motherboard.
- Intel Haswell integrated
Graphics controller
- Intel HD Audio Controller
- Intel Lynx USB Host
Controller
- Intel Lynx Point Audio
Controller
- Intel Lynx PCI Express Root
Port
- Realtek 8168 Gigabit
Ethernet (Highlighted in the image, more about that is given below)
- Intel PCI Bridge
I couldnot
find AHCI controller visible in Devices Passthrough list. I was expecting AHCI will
be available since VT-d was mentioned in motherboard manual. To clarify myself,
I created a support case with Gigabyte and put my concern. I received a
response from the support team. They were suggesting me to check with 3rd party
vendors for AHCI support which they are not having visibility to. I raised query about the VT-d support which they have mentioned in their
manual.
After a
few days I received a reply from the support engineer with a screenshot. They tested the on-board PCI slot with an add-on NIC card and confirmed that VT-d is working fine. We can appreciate the effort that they took to test
with an add-on card. However AHCI visibility in ESXi is not confirmed by them. This may be
visible in future versions.
I double
checked the VT-d support related image provided by vladan and found Asrock is also not showing up
the AHCI controller.
IMHO,
VT-d should be 100% useful when the AHCI is also supported. I confirmed this
from previous B75M-D3H based testing which I have posted here for reference.
Even
though the VT-d passthrough is not working I am not having much impact due to
that. I have another system running with FreeNAS which provides NFS and iSCSI
services to this ESXi test box. I am not having any issue for running my Linux
test VMs from this NAS box. (I will update about this NAS build in my next
blog).
On Board
LAN controller - Additional Testing:
Since
VT-d is allowing me to passthrough the on-board Realtek 8168 chipset, I tested
it by, passing it to a test Windows machine running inside a VM. For all my
surprise, I can make this network card working by installing the driver in the
VM. I used the windows driver available in CD which came with this motherboard.
Moral of
the build:
If you
are looking for a Haswell based Home lab system build and you will keep your
VMs in separate NAS box, Please by a B85 Chipset based motherboard and a
Haswell processor i5 or i7 according to your budget. The B85 chipset
motherboards are capable of supporting 32 GB of memory and the PCI ports found
on these boards can be configured with Intel based add-on network interface
cards for supproting the hypervisors. You can buy an additional memory module
or add-on NIC from the money that you saved yourself by keeping away from Z87
chipset. IMHO Z87 will be a over kill when AHCI cannot be passedthrough to a VM
which may be made as NAS box for nested ESXi host's datastore purpose.
This comment has been removed by the author.
ReplyDeleteAs you have already found out, the Lynx Point SATA Controller that is on many Haswell based motherboards is not supported for passthrough.
ReplyDeleteESXi 5.5 doesn't change this bad behaviour. Do you know if there is any other i7.haswell based Motherboard with a different SATA-controller that supports passthrough in ESXi 5.5? There are XEON-based motherboards that have onboard SATA controllers, but XEON is going in the wrong direction from a price/performance perspective.
Hi,
ReplyDeleteLynx is the chipset provided by intel in its z87 chipset. unless you go for supermicro based motherboars which use sas controllers there will not be any option without the lynx chipset.
On the off chance that you locate that business land, for example land, is accessible in huge pieces which are unreasonably costly for you to get.VR for Real Estate
ReplyDeleteNice post, here's another interesting one real estate marketing 360
ReplyDelete