So I was building out a 44 node vSAN cluster last week and I ran into an issue where 12 of the ESXi hosts had their own network Partition group different than the other 32.  I had no issues with the vSAN network, I was able to use vmkping every server so there was no communication issue with any of the hosts in the cluster  via the vSAN kernel.  In most cases vSAN Network partitioning occurs when there was issue with vSAN kernel communicating with other hosts.

After several attempts of removing the diskgroups, removing vSAN kernel, moving out of cluster and away from DVS then back I had no luck.  I knew based on VMware supported Maximum that I could create a 64 node vSAN cluster.  I was at a loss so after several hours troubleshooting and Google searching I ended up opening a SR with VMware after about hour or so troubleshooting the VMware engineer was at loss until he found an article that indicates that you must create some advanced settings on the ESXi hosts in order to see above 32 nodes and once we made those settings rebooted the hosts we had a single network partition our issue was resolved.

The KB article (2110081) shows how to perform task via esxcli via SSH logged into root but does not show how to do it via PowerCli.

$vcenter = Read-Host "Enter Vcenter connecting to"

Connect-VIserver $vcenter

$cluster = Read-host "Cluster Name"

foreach ($host in (Get-Vmhost -Location $cluster)){
Get-VMhost $host | Get-AdvancedSetting -Name "VSAN.goto11" | Set-AdvancedSetting -value 1 -confirm:$false
Get-VMhost $host | Get-AdvancedSetting -Name "Net.TcpipHeapMax" | Set-AdvancedSetting -value 1536 -confirm:$false
}

Then reboot your hosts.

That’s it!

You would think that supported maximums would occur out of box but according to VMware they did not want smaller vSAN cluster to sacrifice memory overhead that would be required for larger vSAN clusters to run efficiently.