Three Controllers to Rule Them All (that just doesn’t have the same ring to it, does it?)

So we’ve got an NSX Manager now, and the uber-cool nsxcli, so what’s next? We need to build out the control plane, which means it’s NSX Controller time.

We can do this the easy way, or the hard way. We’ll talk about both. But first, let’s think about NSX and what the Controllers do.

The Control Plane is broken up into two main pieces: the Central Control Plane (CCP) and the Local Control Plane (LCP). The Control Plane programs the data plane and maintains the current, or realized state of the network.

That sounds rather similar to NSX-V controller nodes, doesn’t it? That’s because it maintains the same general purpose of the controllers in NSX-V. Except it does more. For example, when DFW rules are published, they are published from NSX Manager to the NSX CCP, and the CCP is then responsible sending the rules to the Local Control Plane for data plane programming.

Another difference is that the interaction between NSX Manager and the controller nodes is that they communicate via a RabbitMQ message bus, not private APIs.

So far, so good. Let’s think about another change: Controller sizing. In NSX-V, each controller had 4 vCPUs, 4 GB of RAM, and (according to my lab) a 28 GB virtual disk. NSX-T, again, provides some sizing flexibility:

  • Small: 2 vCPU, 8 GB RAM, 120 GB disk
  • Medium: 4 vCPU, 16 GB RAM, 120 GB disk
  • Large: 8 vCPU, 32 GB RAM, 120 GB disk.

Why would I use the different sizes, you might ask? Well, Small is generally for labs or proof of concept deployments. Generally, Medium is a good starting point. And Large, well, how big of a shop are you in? You might need it, though.

If you’re looking at your home lab or other small lab environment, you can get away with a single controller, just like NSX-V. Just don’t blame me when something doesn’t work.

For all the general similarities between the Controllers in NSX-V and NSX-T, T introduces some pretty significant under-the-hood changes that have enabled some pretty nice modularity in the Controllers themselves.

So how do we get these things up and running? I mentioned an easy way and a hard way earlier.

The easy way is to just have NSX Manager deploy the entire Controller cluster for you.

In the NSX Manager web UI, go to System > Components, and click on the “Add Controllers” link. NSX Manager will deploy the controller nodes and configure them for you based on your inputs. The downsides to this are that it only works with vCenter, and it only deploys Medium-sized Controllers. You’ve got to go to Fabric > Compute Managers and add your target vCenter server so that NSX Manager has someplace to deploy these virtual appliances. Great for quick deployment, terrible for a small lab environment.

So we have to look at things the hard way for a number of other use cases. Proof of concept, KVM-only shops, big shops that need Large controllers, etc.

To deploy your controller nodes manually, just grab the OVF (or .qcow2, if you’re in a KVM shop) and deploy it. If you’re deploying the OVF, you provide the same kinds of information you provided for the NSX Manager deployment – admin, auditor, and root passwors, network identity, etc. There are some options to add managment plane configuration, as well, should you choose to go that route.

Once they’re deployed, you need to join the controllers to the managment plane. Before we do that, we need the API thumbprint from NSX Manager (we collected that here).

Then it’s a simple matter of running the “join managment-plane” command and passing it the IP address of NSX Manager, the admin username and password, and the API thumbprint. On all of the controllers, we should set the controller cluster security model. Right now, the only option is a shared secret. We run the “set control-cluster security-model shared-secret secret” command to do that.

On the first controller, run the “initialize control-cluster” command to, well, initialize the controller cluster.

From the remaining two controllers, get the certificate thumbprint (“get control-cluster certificate thumbprint” on the controller node). With that information, we head back to the first controller (where we initialized the cluster), and join the other two nodes with the “join control-cluster” command, passing the target controller IP address and thumbprint.

We need to go back to the second and third controllers (one at a time) and run the “activate control-cluster” command. All this work, and we finally have our managment and control planes set up and ready to go. How about that?!

We’re almost ready to build logical network constructs. Almost.

 

~$ history
Introduction: From NSX-V to NSX-T. An Adventure
NSX-T: The Manager of All Things NSX
The Hall of the Mountain King. or “What Loot do We Find in nsxcli?”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s