It seems I have had a number of the same conversations in recent weeks about a very particular topic, and some could argue a design flaw, with the most basic of VMware vSphere deployments. This has to do with using an overly simplistic deployment whereby the ESX host is only configured with (1) VMKernel port, primarily VMK0. While this “works fine” there is some simple inherent issues that can occur downstream with such a basic configuration. Let’s take a short look at the problem, but I am surprised it’s 2022 and some basic fundamentals of vSphere are still mis-understood

Let’s take this simple design shown below and break down what is happening. We need to assume the following is configured as well

• LACP is not configured
• Management, and vMotion services are configured on VMK0 (default out of the box)
• VMK0 has is the only routable interface
• Virtual Switch is configured for active/active of the pNICs

The question we now have to ask is how is traffic flowing for everything in this host? It’s pretty simple really. On boot VMK0 will bind to ONE of the two pNICs based on the assumptions above. This means ALL traffic will flow over a single pNIC including most of your network based storage connections. As you add services and storage the only path to mount these is via VMK0 and thus pNIC1. Are we starting to see the problem? The second NIC is totally “unused”. Now this is a big deal on 1G connection and maybe sometimes even on a 10G or 40G connection. The real issue IMO is “wasting” the other pNIC for any host level services

How most vSphere architects have solved this is simple. Add more VMKernel ports for specific traffic and manipulate the Distributed Port Group teaming settings to re-path the communications. In some cases do this over L2 connections so ensure minimal routing. Here is a simple example of this concept

Lastly if you wanted to expand on this even more here is a table that breaks down using multiple VMKernel ports as shown above I have used many times in past designs. Here you can see we force traffic to active and standby uplinks, (assuming still there is no LACP), thus utilizing both pNIC’s to their fullest extent.

There are other uses for more VMKernel ports for other traffic such as NSX-T, backup, cold migration, replication etc. However, this basic foundational mistake seems to be all too common. Once you are setup in this way other In another post I may show how this can also affect the Storage Migration (Storage vMotion) process as well.

The post The Most Common Mistake Made in VMware vSphere Networking appeared first on Chris Colotti's Blog.

Every year Cohesity, like other companies that support the vExpert Program, try to do something to show their appreciation for the community.   In the past there has been backpacks, last year was some tee shirts and charging portfolios.  Every year it does get harder and harder to come up with something different not only for the item but for the method to claim the item(s).  This year STK Promotions, who many in the community know, came up with a cool item that falls into the “car” theme the Technical Advocacy Group has been using all year.

Now I can’t give up the item’s but there has been a lot already claimed, and I admit this post is a day or so late, but there are still some left.  However at the same time I want to thank so many for all the tweets, direct messages, and the feedback provided on the process we used this year.  Here is just some of the comments.

I love this style of engagement. I think it’s educational and fun – and I think you’ve balanced it well so it’s not tedious. Great job!

this was GREAT! I had fun learning more about Cohesity than i have before. Great work team, very interactive and fun giveaway. I felt like i was back in the late 90s and early 2000s running around the web clicking on links to get FTP login info to download MP3s and Warez!

Well i have been hearing about Cohesity for last one year after few connection from vExpert community. I got so know something about this product and its nice to know about Mr. CEO. If I get chance i will definitely explore more into it.

This was amazing and so much appreciated. Thank you for creating this. It was educational and a super fun course

In addition to this there has been a lot of chatter on twitter as well.  I am humbled that so many people not only appreciate the item but also appreciate that programs like this come at a cost.  Not only for the item(s) themselves, but with the Pandemic we also have to figure out a way to ship all these individually, thankfully STK Promotions was there to help with that part.  I will be doing a vBrownBag session on how much of the backend was automated next week.

Claim Yours Now!

So with that, there is still a few left so if you read this post amidst your busy day, grab one up and enjoy.  Click the image below for the website.

The post The 2020 Cohesity vExpert Gift Progam appeared first on Chris Colotti's Blog.

Late last year Tintri announced the new EC6000 and T1000 platform to customers.  One thing that I have noticed in some of the various support tickets is that many of the root cause issues have actually been upstream networking issues, or simply a lack of networking validation at the time of or ongoing after deployment.  Like anything these issues can be easily prevented with a better understanding of how the networking in the new Tintri EC6000 and T1000 series works.  I am also going to apply some personal best practices here as well.  If you have not figured it out the EC series and the T1000 are based on the same platform chassis, hence the networking is identical.

The Tintri Networks Available

First let’s take a look at the networks that are available on these new platforms.  I will cover the most common of them, and the physical connectivity of the PCI cards can vary but the usage of them will remain the same.  Below is a back panel diagram illustrating the most commonly used ports.  You will note that each controller has a duplicate interface configuration.  Other configurations may be done, but this is a pretty common setup.

• IPMI Port:  Remote access capability using IMPI command line tools
• Management Interfaces:  Used for UI/API access and used by TGC (Tintri Global Center)
• Data Interfaces:  Used by hypervisors to mount the NFS storage
• Replication Interfaces:  Dedicated for Tintri –>Tintri or Cloud Connector replication

Each interface can be configured using VLAN trunks to provide different VLAN’s on each set of interfaces or they can simply use access ports and each of the four be bound to a single VLAN.  This is entirely up to you based on your requirements.  Going forward let’s assume these are using access ports for simplicity.

How The Tintri Network Pairs Behave

There are two different ways these port pairs can behavior depending on if you configure them for LACP or not.  Out of the box LACP is not enabled but can be in the VMstore settings.  For the sake of simplicity let’s assume LACP is not enabled going forward.

Here we can see that each pair has a Primary/Standby Designation.  Generally speaking the “left” or “Top” ports are the Primary connection.  The other way to identify this is by the lower letter designation.  This is important because the Tintri OS is configured to always use the Primary connection path if available.  So in a non LACP configuration this means even if your primary switch fails and comes back the network will move to the standby port and then back to the primary.  In an LACP configuration this would not be the case.

It is also important to understand that if these ports are properly configured the system will NOT fail over the controller just based on the loss of a network path to one of these ports.  If the controller does failover, the IP addresses are moved over to the other controller and the behavior would be the same.  Where this becomes problematic is if the access port VLAN is mis-configured at deployment or later on down the road.

Proper Cabling Of Tintri Network Interfaces

Based on the above information, and again assuming that we are not configuring for LACP, the following diagram shows the proper network connections.  Again this is also assuming VLAN access ports are in place for simplicity.

Really, this is very simple.  Primary interfaces are connected to Network Switch A, and standby are connected to Network Switch B.  These could be top of rack switches or other edge switches.  You would connect both controllers the same way since only one controller is ever active at the same time.  This configuration would ensure that if a single port failed, or the switch failed all traffic would move to Switch B without a controller failover.  During a controller upgrade, connectivity would also remain stable.  You could make this more complicated by reversing the connections on Controller B, but that is up to you.

If we did happen to enable LACP we simply need to be sure that the two switches can support cross-switch LACP.  In my personal lab this is not available to me I can only do LACP on ports that exist on the same switch.  In that case I could cable Controller A to Switch A and all of Controller B to switch B.  However that would require a controller failover just for a switch failure.  Again, depending on your switches, and your requirements that may be something you want/have to do.

Testing Of Tintri Network Interfaces

Long ago I wrote about the importance of a test plan on vSphere hosts, especially those with many VLAN port groups.  Tintri has long said that their design mimics that of vSphere hosts, so testing the networks really is no different.  Assuming we stick with the basic configuration above the test plan is fairly simple and can be done with basic ICMP Ping tests.

• Continuous Ping admin ports
  • Pull primary port cable (Observe failover, a few dropped pings)
  • Re-connect Primary (Observe failback, a few dropped pings
  • Controller should NOT failover unless both cables are removed
• Repeat above for replication ports
• Repeat above for data ports
• Manually failover controller and repeat all three tests
• You can also fail an entire switch or down the ports to simulate
• Finally test an automated controller failover by removing 2 of the same ports which simulates an internal card failure

In theory these tests can and should be performed regularly as changes do happen on the network.  If you ensure all this, you will have no issues during controller upgrades that involve the failover of the controllers to perform the upgrade.  You should also have no connectivity issues from inter port/card failures.  The best defense is always a good offense.  Hopefully this will help customers and partners as they deploy the new EC6000 and T1000 platform.

The post Understanding The Tintri Networking Connection Options appeared first on Chris Colotti's Blog.

I realized today that I have not actually blogged yet about the Tintri Synchronous Replication feature that was released a few months back.  I decided to jump back into the lab to show people just how easy this is to use.  What Tintri has built it’s base on has always been ease of use, and to be honest this feature is no different.  Let’s take a look at just how easy it is to configure and use.

The Basic Steps

For you to configure and use Tintri Synchronous Replication, like the SRM setup I detailed previously it’s a few simple steps.  This will require the use of Tintri Global Center 3.7 or higher to implement.  I suggest you read the Tintri Global Center admin guide for latency requirements and other considerations.

• Create a Tintri Service Group using “Option 2”
• Mount the sub folder created on vSphere hosts
  • Use the Cluster IP and sub folder path
• Move or deploy virtual machines to the datastore
• Monitor the groups replication in Tintri Global Center

Tintri Service Groups

The first thing to look at is something I wrote a couple of weeks ago to understand the difference between the Tintri Service Groups.  The reason is there is some differences between how they are used and where they are configured.  For the sake of this I will assume you’ve got the understanding of the different ones.  Bear in mind Tintri is hoping to converge these in the future to make things easier.  The first thing you need to do is create a new Tintri Service Group and run through the six simple steps.  Remember this Tintri Service Group today cannot be used with SRM those are configured separately.

First just give the group a name

Then select “Option 2” for the synchronous replication

The next step is where you define a sub folder for the group to use as well as the PRIMARY VMstore.  The folder will be created and we will use this later to mount as a datastore in vSphere.  This folder will be created on both Tintri VMstores from this one screen.

Here is where we setup the cluster configuration.  We select the secondary VMstore and we assign a cluster IP address which should be on the Data network range.  This cluster IP will be used to mount the previous sub folder as a new datastore.  In my lab only one VMstore as a Replication IP, but ideally you would use both replication networks on both ends.  in my case I am using the data network on the other side.  Then you can test the settings.

We can just set some basic alerts

Finally review the settings and create the Tintri Service Group

Once this is done the group will be created and do an initial setup.  You can see the status of the group to check or modify settings or add snapshot protection as well to the group.

Using the new Service Group in vSphere

As I mentioned at the start once you setup this new “Cluster” you just need to present the new construct to vSphere.  To do this you simply mount the datastore as a new NFS mount point.  You will use the Cluster IP you created and the sub folder.  I suggest you name the datastore to reflect the Tintri Service Group’s usage.  This new datastore will effectively span both Tintri VMstores with the single namespace.  While you can still deploy virtual machines to the base Tintri mount point those will not be replicated.

At this point you can migrate machines or deploy new ones to the new datastore and they will begin replication to stay in sync.  One thing I will point out is that if you try to browse the sub folder on wither VMstore mounted directly in the default folder you won’t see anything.  This is by design which is access denied, and the only way to see the virtual machine files is via the new datastore.  At this point there is nothing you really need to do.  You can pause replication or failover from the group settings.  Here we can see there is one virtual machine in my new service group which was deployed to the new datastore.

The post The Ease Of Setting Up And Using Tintri Synchronous Replication appeared first on Chris Colotti's Blog.

This is just to serve as a quick update on the Tintri vSphere Web Client Plugin that was updated last week.  The plugin has been out for a while and many people use it to take advantage of Tintri specific actions from within the vSphere Web Client.  It’s been very stable and Tintri added in a few more features to it with the latest patch release.  I will note this is still the flash client plugin and Tintri is working on an HTML5 version.

Tintri vSphere Web Client Plugin

If you have not installed it before the plugin is pretty easy to download and install from the Tintri Support Portal.   You should give it a go, but here is the new enhancements of the plugin.

• Tintri Cloud Connector: View and restore Tintri snapshots taken using the Tintri Cloud Connector.
• Synchronous Replication: View the synchronous replication status and path for VMs.
• Privilege for Refreshing Virtual Disks: Added a privilege, “Refresh Tintri Read Only Virtual Disks,” to give read access to VMs from which to refresh data.
• Multiple Replication Paths: Displays the replication status and path to all relevant VMstores.
• Simplified Folder Creation: When adding a datastore, automatically create a directory for the user if the mount path does not exist.
• Right-click to Delete Snapshots: Delete snapshots by right clicking on a VM.
• Restore VMware Best Practices: Restore VMware best practices when users want to revert to previous values.

Some of these functions are still initiated by Tintri Global Center or the VMstore UI, but now you can view and use some of these from within vCenter.  One example is the Tintri Cloud Connector capability to download the snapshots from the cloud without having to go to Tintri Global Center.  Just highlight the Virtual Machine that you want to explore and go to view snapshots

From here you can see all the snapshots including the local and cloud based options.

You can then simply select a cloud version to start downloading and which VMstore to download it to.

Here we can see that the 8:05 version from Jan 30th is now also stored as a local copy.  We can now simply clone this back to a Virtual Machine and we have never left the vSphere Web Client.

Try The Tintri vSphere Web Client Plugin

If you are currently a Tintri customer and you are not running the plugin, give it a go.  If you are doing a POC, then add the plugin into your testing so you can see how it works.  It does give easy access to some of the most common functions and makes it easy to manage multiple VMstores from the vSphere Web Client.

The post Tintri vSphere Web Client Plugin Updated appeared first on Chris Colotti's Blog.

There has been a lot happening to Tintri Global Center recently and the latest release is no exception.  I wanted to show you one quick major feature update that was included.  It is the ability to update your managed VMstores right from the TGC interface.  In my opinion this added feature marks the first major milestone for TGC to truly become a central management interface for multiple VMstores.  Past features for cloud connector, expanded service groups, and storage migration offload via storage pools were added back in 3.7.  If you manage multiple VMstores you want to upgrade to Tintri Global Center 3.8.

Upgrading VMstores from Tintri Global Center

The first thing I want to point out is the upgrade process and steps performed still remain the same.  If you are not familiar with how a VMstore performs OS upgrades here is the basic flow.

• From laptop or admin console download new TXOS RPM bundle from Support portal.
• From VMstore UI, settings, more, upgrade, then the path to the bundle
• VMstore will check new bundle and compare to current TXOS version
• TXOS will upgrade standby controller, then reboot upgrade controller
• TXOS will automate a failover,
• The previous active is now standby controller, goes through the upgrade a second time, reboots, then remains standby

The only steps that are now different is you can upload and manage the upgrade in one place instead of uploading to multiple VMstore User Interfaces.  Let’s check out the changes.  To begin below is my T1000 showing that controller B is in standby

next we can log into TGC and select the VMstore option in the menu and see the “Upgrade” feature.  Once there we can also see the VMstores managed and their OS versions in one place.

Assuming we have downloaded the latest Tintri OS version we can now upload it and the package will be extracted for use.

Once it is unpacked you will be able to see that the package is valid and you can begin your upgrades by selecting the VMstore(s) and clicking “Upgrade”.

At this point the package will be uploaded to the VMstore and the process will continue as described above and you can see some of the process if you expand the VMstore details.

Once the process is 100% completed you can see that the previous standby Controller B is now active and both are the same version.

The post Tintri Global Center 3.8 Goes GA With VMstore Update Feature appeared first on Chris Colotti's Blog.

A few weeks ago I wrote about how easy it is to set up Tintri Synchronous Replication groups.  Last week I was at some customer sites not only discussing the feature but talking about various use cases for it.  There are a few that came to mind but I wanted to show one basic, yet viable solution.  The idea stemmed from a simple question from a customer that stated “If one Tintri is so awesome, and we love it, What can I do with two Tintri’s?”  It’s funny how such a simple question can spawn a 45 minute white board working session.  Here is a little bit about one of the uses cases we talked about.  The concept is building a basic pod based architecture to allow for easier operational maintenance and changes.

“Why are two Tintri’s better than one?”

If you refer back to the article about setting up Synchronous Replication the basic concept is that the two Tintri units are accessed via a cluster IP that resides on the data ports.  You can create multiple cluster IP addresses which can enable this pod based architecture design.  The basic components would be two racks configured identically.  For the purposes of illustration here is what I will use going forward

• 3 vSphere hosts per rack
• 2 top of rack switches connected per This Article
• redundant power
• 1 Tintri EC6000 per rack

The concept here is to build a vSphere DRS cluster than spans the two racks.  As you can imagine this could also be done across campus or across town should the latency support it.  The idea shown below setup simply with two Tintri cluster IP’s each running on the local side Tintri as the primary.  These are presented as separate single datastores to the cluster.  So in effect the cluster in this case would see TWO simple NFS datastores.  The are hosting virtual machine for each half of the cluster using DRS ‘Should Run’ rules to insure they try to stay locally on their half of the cluster.  This provides locality to the hosts on that side of the cluster.  All the while both Tintri Sync Replication groups are replicating back and forth.

Once this is configured and we need to do any kind of POD based maintenance we can put the hosts in one cabinet into maintenance mode.  Due to the “Should Run” run in place, it will allow these to be evacuated to the other hosts in the cluster.  If we only put a single host in maintenance mode they would try to remain on the left side.  This is also assuming you have architected the POD to take a one-half outage for this purpose.

Once the vMotion is completed the hosts on the right side would be accessing the primary Tintri in the left cabinet.  For a localized architecture this should not have much it any performance hit.  If this was across town or campus that could be a design consideration in how long you run this way.  Finally, if you invoke the manual failover of the Tintri Sync Replication group you would move the data path completely to the other side so now in fact you could do complete maintenance of the left side of the image, as everything we be free of traffic.

Does This Qualify as a vSphere Metro Cluster?

At this point technically no it does not primarily because the manual failover does not yet have an automatic option to work with Metro Cluster HA.  However, Tintri is working on this as part of adding “automatic failover” to the sync replication groups in an upcoming release.  At that point, we can test and demonstrate the High Availability portion of the design.  Once both the DRS and HA aspect of the cluster can be fully supported and designed to work as a metro cluster, then you can call it that.

However, even at this stage the setup and simplicity of the replication group and the clusters to work at this level is very attractive.  This design can be stood up in a very short time and utilized not only for this use case but a couple of others until the full Metro Cluster HA is made available.

The post Tintri Use Case: Synchronous Replication Pod Architecture appeared first on Chris Colotti's Blog.

Note:  EVERYTHING IN THIS POST IS UNSUPPORTED BY TINTRI

After I did some lab work with the Minio Gateway using NFS it occurred to me that something similar could be done but using a Tintri VMstore as the target for Docker Volume persistent Storage.  While this seems easy on the surface it has a lot of considerations which is why it’s not something currently supported by Tintri, but it does in fact work.  The premise is built on the creation of a Docker Volume that is mounted to an NFS share first then used to attach to a container.  Most people know that Tintri uses NFSv3 to connect so I found that this does work but has some very strict considerations.

Docker Persistent Volumes on Tintri Considerations

1. You should always create a quota based Tintri Service group to ensure you do not consume all the available storage on the array.  This is a safety net for the most part.
2. You can only mount with the “nolock” switch since Tintri is NFSv3 based so you should create other subfolders in the service group folder and mount each of those individually to prevent potential lock issues.  Without locking you cannot have multiple containers access the same data for any large scale use.  This problem is actually solved with NFSv4.
3. These Docker Volumes not able to take advantage of advanced functions like snapshots, but I’m not able to tell if de-duplication or compression could be applied but could very well come into play behind the scenes.
4. The Docker host must have access to the Tintri Data IP address
5. Since this is not akin to a vDisk and it is a directory tree of files, it’s worth warning against using large numbers of small files.  The Tintri filesystem is optimized toward fewer, large files rather than billions of small files. Large numbers of small files could cause some internal resource pressure on supported workloads.

Assuming you have network access the easiest way to test this if you don’t have a docker host is by deploying PhotonOS as in my previous post.  The commands are very similar with the exception of creating the Tintri service group and subfolders first.

Create Tintri Service Group With Quota

Using Tintri Global Center (TGC) click Service Groups and click New Service Group and choose option 3

Select the VMstore you want to use and then assign a sub mount folder name for the group.  It will always be a sub folder to the root “/tintri” folder

Once you complete these steps you can assign the quote to the group to protect the storage from being consumed too much.  Simply edit the settings and add a Quota of your choice.

At this point you can mount the folder and create additional sub-folders to use for your individual Docker volumes.  Some people may be able to mount these with a standard NFS client, but for some of you these may only be visible through the vSphere Datastore browser so that works too.  You can see below I’ve created two sub folders for a Unifi and NGINX container.  These folders will be passed into the Docker host to be used as persistent storage for these containers.

Now we can execute the Docker Volume commands to set up the mount points to this Tintri VMstore and the associated folders.

docker volume create --driver local --opt type=nfs \
   --opt o=addr=<Tintri_DataIP>,rw,nolock \
   --opt device=:/tintri/Colotti-Docker/unifi unifi

docker volume create --driver local --opt type=nfs \
   --opt o=addr=<Tintri_DataIP>rw,nolock \
   --opt device=:/tintri/Colotti-Docker/nginx nginx

Then we can deploy the containers themselves inserting the new volume as the persistent storage

docker create \
  --name=unifi \
  --restart unless-stopped \
  -v unifi:/config \
  -p 3478:3478/udp \
  -p 10001:10001/udp \
  -p 8080:8080 \
  -p 8081:8081 \
  -p 8443:8443 \
  -p 8843:8843 \
  -p 8880:8880 \
  -p 6789:6789 \
  linuxserver/unifi

docker create \
  --name=nginx \
  --restart unless-stopped \
  -v nginx:/config \
  -p 80:80 -p 443:443 \
  linuxserver/nginx

Once the container is deployed you can refresh the vSphere datastore browser and see that the configuration data is now in that folder and can be mounted later to ensure that changes are kept when a new version of the container is deployed.

What is cool about this from my PhotonOS post is that all the NFS mounting is being controller by Docker with no need for NFS-Utils or other package requirements on the PhotonOS virtual machine.  This approach is all within the constructs of Docker and both of these containers are bound by the 500GB quota of the Tintri service group as you can see from the console of the NGINX container.

Conclusion of Docker Persistent Volumes on a Tintri VMStore

I need to say this again that this is UNSUPPORTED by Tintri, but I’ve proven this does in fact work and could have some cool potential use in the future.  Containers and their data could be treated like any other virtual workload and getting them on the filesystem is one thing.  Applying all the advanced Tintri functions to them is a completely other animal.  Bottom line is this does work and it is cool to play with in a lab, especially if that is where you are playing with containers anyhow.

The post How To Use A Tintri VMstore For Docker Persistent Volumes appeared first on Chris Colotti's Blog.

Last year Tintri released the feature known as Cloud Connector which is used to offload snapshots to S3 based storage providers for long term archival.  Since the feature came out the majority of the use cases I have been on have been for on premises solutions versus using actual Amazon S3 in the public cloud.  Primarily this is because people have some older storage on premises they want to re-use for S3 bucket storage.  The solution is to use Minio as an S3 gateway in Docker, running on PhotonOS, mounting NFS based storage for fast, easy S3 offload.

Configure PhotonOS for Minio Gateway

This is probably the easiest thing I have done in a long time.  Head over to the PhotonOS download page and grab the latest image you need.  I grabbed the VMware version but what is cool is there is plenty of other options.  Once you import it and deploy it and set a static IP, run the following commands

tdnf update
systemctl enable docker
systemctl start docker

What I was taught by a co-worker was that Docker is already on PhotonOS which makes this so crazy simple.  Once Docker is running here is all you need to do, assuming you already have an NFS export in place.  Now there is a few things I found but NFS Version 4 seemed to work the best.  Some of my NFS shares on certain devices had locking issues with version 3.  This will create a Docker Volume that will attach to the NFS mount.

docker volume create --driver local \
    --opt type=nfs \
    --opt o=addr=<ServerIP>,rw,vers=4 \
    --opt device=:/<MountPoint> <VolumeName>

# Optional for NFS Version 3 with no locking option 
docker volume create --driver local \
    --opt type=nfs \
    --opt o=addr=<ServerIP>,rw,nolock \
    --opt device=:/<MountPoint> <VolumeName>

Deploy Minio Gateway with NFS

Next you just deploy Minio using that Docker Volume for the gateway mode

docker run -d --name <ContainerName>  -p 9000:9000 --restart unless-stopped \
    -e "MINIO_ACCESS_KEY=<YourAccessKey" -e "MINIO_SECRET_KEY=<YourSecretKey>" \
    -v <VolumeName>:/<ContainerMountPoint> \
    minio/minio gateway nas /<ContainerMountPoint>

Once that is done you can access the Minio webUI at http://<PhotonOS_ServerIP>:9000with the access and secret key.  Test creating a bucket and uploading files and if that all works you can attach to Tintri Cloud Connector and test with snapshot offloading.  Obviously there are some other options you can deploy this with but there is no faster way to get an S3 gateway up and running and Minio supports more extensive deployment models as well.

It also stands to reason that you can do this to run a front end gateway for Google Cloud Platform and Azure as well.  I may test those in the future but the basic concept would be the same.

The post How To Configure Minio Gateway on PhotonOS With Tintri Cloud Connector appeared first on Chris Colotti's Blog.

The interesting thing about having worked with VMware for so long and now for Tintri is that you see some of the same issues in different environments.  That is what prompts most blog posts about “how to” fix or address things, and that is no different for me.  In the last couple weeks I have been involved in a few cases where some of the basic vSphere principles as well as some of the long-standing NFS best practices just are not followed.  In my opinion as things have gotten “easier” to install and manage, we tend to forget a few of the key things to prevent issues in the future of the architecture.  This is my short attempt to address some of that.  A lot of what I am going to cover is in the Tintri Best Practices Guide.  I will pull out some key points from there as I go.  I hate to say it but sometimes we need to get back to basics.

If you have not read the guide but want a primer on the physical networking aspect, you can refer to this blog post that covers that specifically before we get into some of the vSphere specifics.  What I want to address is some of the core basics that people seem to forget.

Preparing The vSphere Hosts

There are a few things you always want to do on the host side to ensure you are ready to connect a new Tintri VMstore to your environment.  For many this may be the first NFS device they are using and may be switching from iSCSI or Fibre Channel.  You want to be sure you do the following:

• Install the Tintri VAAI Plugin
• Set the NFS advanced parameters per KB article 2239
  • Nfs.MaxVolumes = 256 (set to max for version running)
  • Net.TcpipHeapMax = 1536
  • Net.TcpipHeapSize = 32

All of the above require host reboots so before you do one you might as well do them all.  The settings above are for vSphere 6.0 and later versions the numbers could be higher.  The key is that both of these values represent the memory allocation amounts in MB.  I can tell you from personal experience, on hosts built as older versions that were then UPGRADED to 6.0 or later, these settings can in fact be lower, in some cases set to 0 and will in fact present host latency that the Tintri UI will actually point out.  It’s better to set these before you even start then have to go back later.

Consider Your MTU Size

Let’s move on to the host networking as that is also where I see a lot of back to basics conversations happening around vSwitches, MTU size, and teaming policies.  In general most people working with network storage understand Jumbo Frames and now enable it on the physical switches.  You can also enable this in the Tintri device itself and that is recommended.

Assuming you have this set on the physical switches and the VMstore you want to double-check not only the vSwitch level.  I understand maybe this is trivial to point out but that is why this is about back to basics.

Consider Port Group Teaming and TCP/IP Stacks

This is typically similar for most people these days that setup hosts.  There is some agreement in the community on acceptable number of vSwitches or physical NIC ports.  In most cases this is driven by what you have on your hosts to work with.  Below is a typical setup based on 4 Physical adapters.  The key sometimes comes down to the teaming policy overrides which sometimes I find are not done.  This assumes that the physical NIC’s are dual port separate cards.  Now this may change for other configurations but the KEY is the teaming override.

#vswitch0
    vmnic0,vmnic2 (active/active)
vmk1 (management) default TCP/IP Stack
    vmnic0,vmnic2 (active/standby)
vmk2 (vMotion) vMotion TCP/IP Stack
    vmnic2,vmnic0 (active/standby)
Virtual Machines
    vmnic0,vmnic2 (active/active)

#vswitch1
    vmnic1,vmnic3 (active/active)
vmk3 (NFS) default TCP/IP Stack
    vmnic1,vmnic3 (active/standby)
vmk3 (NFC) Provisioning TCP/IP Stack
    vmnic3,vmnic1 (active/standby)

The reason the override is important is on kernel ports if both are set to active/active you will not know which one the kernel binds to on startup.  Also assuming there is no LACP configured here, you want to isolate the traffic manually.  This is regardless of standard or Distributed Switches.  By specifying the different TCP/IP stacks in later versions of vSphere you also define the traffic types.  Many people do not even add a kernel port on the “Provisioning” stack and therefore all network File Copy, clone and other operations will usually default to the management port causing some management hiccups at some point.  By defining the vMotion and Provisioning stacks you ensure the traffic for those will use those kernel ports.

It also goes without saying each of these should be a separate VLAN and IP subnet.  I have seen environments where the VLAN was different but the same IP range was used on the kernel ports and communication was not functioning.  It costs nothing to add VLAN tags and create a new IP range so might as well do it.  If you have more than 4 ports you can create a third vSwitch just for Virtual Machine traffic, but in some cases you may only be working with two physical links.  Either way work with what you have.

Connect To Tintri via NFS

Once all of this is set up THEN you are ready to mount the Tintri storage via NFS on your hosts.  Okay, you can probably mount it anytime and it will work, but my feeling is you don’t just want it to work, you want it to work near perfectly.  I can tell you that when I get calls about issues, the first places I look are the NFS heap settings and the physical and virtual networking.  Most of the time the issue will be found in those places because let’s face it….Tintri just works.  It works even better if you stop and take a few minutes to simply review your environment first before going all Leroy Jenkins adding storage and moving Virtual Machines Around.

The post Configuring vSphere Networking With Tintri Storage – Back to Basics appeared first on Chris Colotti's Blog.

There have been a few posts I have seen recently on folks making moves around the industry so I guess it’s time for mine.  I don’t think it is any secret to people that I am moving on from Tintri after this week.  The last year was certainly a learning experience for more reasons than I think most people would care to know.  At the end of the day the Tintri technology truly is some of the best out there.  It does what it does very well.  Sadly the recent months, with the Bankruptcy filing has left many looking for other opportunities.  While I was spared both times, once in March and again in June, I decided it was time to take control of my life again.  I can honestly say I would never wish that anyone have to see the downward spiral of a Chapter 11 first hand.  It’s not fun, it’s a lot of stress on you, your family, your co-workers, and your customers.  In this case specifically just watching how some of it unfolded and was handled for me personally was just enough.  It has come time to close that chapter of my life like many others I worked with at Tintri that were some great people.  So what is next?

The funny thing is I was considering a lot of options from working at Lowes, to focussing on some other business ideas to working with Julie full time on STK Promotions.  The day that Tintri announced the major layoffs I was actually let go before lunch like everyone else, via E-Mail.  I made one single post on LinkedIN that over the course of the week ended up ave over 23,000 views.  However, about 4 hours after being let go, I was called and told I was a mistake.  I was still being kept on and I was employed along with about 50 other people at the time.  By that time it was not worth removing the post as it started to at least generate some interest.  The funny thing was that I was starting to question my own value around the industry.  Let’s face it, I’m 44 going on 45 and not getting younger.  I think I have said before in posts that learning gets a little harder to do as I get older.  There is new things coming at us all the time and sometimes it gets harder to keep up.  I was truly not sure if I was going to be able to find another job after having just left VMware over a year ago.  This was never my intent to stay somewhere for a year and then move on, so I was actually worried about myself and my family.

What I  learned in that week was that my network of people truly is a family.  There were a lot of people that reached out just to wish me well, but more people came forward with ideas and places to go look along with introductions to new people, recruiters, and headhunters.  I started to think, maybe this will not be so bad or as hard as I thought it would be.  At the end of two weeks of E-Mails, phone calls, texts, and direct messages I ended up having two offers to chose from.  It was not an easy decision but I decided to join Cohesity working with Aaron Delp and his team.  It’s a bit of a mixed role, which I like as it gives me from freedom to work on many things.  I have to thank Aaron and Rawlinson for being there from the time of that first post about being let go, and coming to the table to talk right off the bat.  I am in the same boat as last year starting a new role with a new company and new technology to learn.  That should not take long, but I am excited about the opportunity to keep working on some cool stuff.  I always want to be a person that is able to have an impact and I think this is a place where I can for some time to come.

What comes after this is as always a mystery of course.  I have some short term and long term plans I’d like to achieve for sure, and some day I would like to do something completely different.  Until then, onward and upward with Cohesity and I will once again be at VMworld in a different color shirt.  I am truly blessed and humbled by the response to the situation and the conversations I had to get to a resolution.  I do wish Tintri well in their rebuilding process, and I hope at some point it can move forward with a great technology, I just personally need a little more stability than what is there now.  I wish the people there the best and I even told some of them to give me a call again when things get put back on track.  You never know what can happen in a year or two.

The post One Door Closes as Another One Opens appeared first on Chris Colotti's Blog.

Okay, so it’s been just over 3 weeks since I joined Cohesity.  Last week I was finally able to get a bit hands on with the Cohesity Virtual Edition (VE) in my home lab to record some videos for VMworld.  Yeah I got tossed right into that even with only a couple weeks on the payroll, but hey I like a challenge!  I wanted to give a little bit of my personal experience just getting things setup for the first time.  Granted working with a full cluster setup has some differences, but I wanted to at least cover the deployment of VE, and go over some of the things I found very easy to do thanks to the improved user interface.

Cohesity Unified Interface

The most striking part of any Cohesity deployment regardless of physical cluster, Virtual Edition, or Cloud Edition and what impressed me the most day one, was the fact that all version have a unified interface.  In fact during Cloud Field Day 4 the user interface was a point of note by the delegates.

It might be the UI that gives @cohesity an edge. They’ve found a way to surface the features in a way that makes sense. #CFD4

— Nathaniel C Avery (@nathaniel_avery) August 10, 2018

This simple aspect, in my opinion makes things much easier for the administrators.  Not only is every edition the exact same but you don’t have to go hunting for something from one version to another.  You can even cross manage deployments my linking remote clusters together for management or replication.  Below is an example with not much data from my private lab running the Cohesity Virtual Edition.

About The Cohesity Virtual Edition

The first thing to understand is that the Cohesity Virtual Edition (VE) is intended for remote or branch offices and is available in both Hyper-V and vSphere formats.  The idea is that you do some local backups to the appliance, but replicate those back to the main Cohesity Cluster back at Headquarters, OR utilize the CloudArchive feature to simply push them to a cloud based object store for later retrieval by another cluster.   While it still gives you some local retention on the appliance itself you can leverage the features of CloudArchive, CloudSpin, CloudRetrieve, and CloudTier the same as you would for any other Cohesity cluster.

Deploying Cohesity Virtual Edition

The appliance is available in two sizes from the same OVA.

• SMALL configuration supports a Virtual Machine with 4 vCPUs, 16 GB of memory and a 64 GB virtual disk to store the Operating System
• LARGE configuration supports a Virtual Machine with 8 vCPUs, 32 GB of memory and a 64 GB virtual disk to store the Operating System

Each version will require adding TWO additional Virtual Disks for the Metadata Tier and the Data Tier.  These sizes are well documented on the Cohesity support site when you access the appliance so I won’t go into detail here, just know you have to add these two disks before you power it up.

Each Appliance also can be configured with two network interfaces and does support VLAN’s on the Primary interface

• Primary: Data
• Secondary:  Management

In the Virtual Edition documentation you will read that if you configure the secondary interface it will be assigned the default gateway and be used for replication to a remote site as it will have the default route.  You can segment the traffic on the other interface using additional VLAN tags as needed.  Remember this is meant for remote and branch offices and there should not be that much traffic.  At a minimum I would probably configure the secondary for the replication portion to send it over a different interface.

Setting Up Cohesity Policies and Jobs

Seriously, this is the easiest thing to do for sure.  You can setup your policies the same as the physical clusters to replicate to remote destinations, and then simply apply those policies to any machines, views, or other objects.  The most impressive thing is how all data on the platform is treated similarly.  The exception is Virtual Machines that are even completely indexed for later file and folder search.

Once you setup your policies searching and browsing for items to apply the policy too is also very simple.  You can search by name, or expand and simply select what you want to add to the backup job, and then assign the policy you want.  The thing to remember is it’s Policy first, then job, but you can even create new policies in the context of creating the job without backing out of the screen and starting over.

Also something that did not go unnoticed during Cloud Field Day 4

Something that seems small, but goes such a long way as a user. The number of times I've needed to stop a workflow because of a missed setting …#CFD4 https://t.co/uZCw0idyAF

— Matt Crape (@MattThatITGuy) August 10, 2018

Setting Up Cohesity Cloud Targets

Everyone knows I have been a cloud guy a long time, but I have to say I was completely impressed with the number of targets available for CloudArchive and CloudTier functions within a policy.  Just look at the list below out of the gate which is more than other products I have worked with that claimed “Cloud Archive” capability but to only one provider type.

Cohesity First Impressions

Let’s face it I work here now and next week I will be at VMworld in a Cohesity shirt.  However, I am very impressed thus far with some of the things I have seen.  I am not one of those people to go into something expecting everything to look right, or even work 100%.  Sure my nature is to always find a way to improve something and that will come, but out of the gate this still is amazing and it works.  Come see me in Vegas and let’s talk about Cohesity!

After only a few days of seeing some stuff going on @Cohesity HQ all I can say is…. pic.twitter.com/LOd6iV0rbh

— Chris Colotti (@ccolotti) August 9, 2018

The post First Impressions and Hands On With Cohesity appeared first on Chris Colotti's Blog.

It seems like not to long ago I wrote one of these posts when I started with Tintri.  Today literally wraps up my second week on the job and I can say this place truly is a rocket ship.  I heard people refer to it that way but I didn’t really understand it until I came out to HQ and saw it for myself.  Having only really changed jobs a few times in my IT career, I don’t have a lot of comparisons to make, but the ones I do have even from my most recent company to this one are astounding.  Outside of the people and the culture, the insane pace that Cohesity moves is head spinning.

After only a few days of seeing some stuff going on @Cohesity HQ all I can say is…. pic.twitter.com/LOd6iV0rbh

— Chris Colotti (@ccolotti) August 9, 2018

I think that tweet pretty much sums it up for real.  Within days of getting into the office I was humbly asked to jump into about 10 things for VMworld, Ignite, first half planning, I mean it’s awesome and crazy all at the same time.  I was sitting in a meeting today and someone said “how long have you worked here?”  I said about 2 weeks, and they responded saying, really?!  I’d have thought you were here for years!  There is no other way to describe how that makes you feel right at home.

While I am wrapping up trying to get back home, I will say that I can tell this place is going to be very cool.  I can’t wait to get out and start presenting on the technology and the use cases it can solve.  Some of the stuff we covered today in Cloud Field Day 4 is just amazing.  Not only how it works, but how complete the features are even early on.  If you did not see the live stream go check out the recording and hit me up with questions.

Once again I will be at VMworld in a new color polo, but hey, it’s all good!

The post Jumping On Board The Rocket That Is Cohesity appeared first on Chris Colotti's Blog.

Not only am I still new to Cohesity, but I’ve honestly never even touched Microsoft Azure until this week.  As Jon Hildebrand and I have been preparing a live demo environment for Microsoft Ignite, we decided we needed to build a Cohesity Azure Cloud Edition.  What I wanted to do was put into some simple terms the steps needed to perform, and some of the things to keep an eye out for that took me some time and multiple tries to realize.  Some of it was human error, others were aspects of Azure itself.  The other cool thing now is with Helios you don’t technically need a VPN to manage the Cloud Edition Cluster and I will get into that in a bit.  I will make a point to say I did this with a single node but production use requires a minimum of THREE nodes.  I will point out where I made that change.  You will need access to the Cohesity support portal for download links and a valid license key.

First Check Your Azure Quotas

I ran into this later on, but it is one of the first things you want to check.  Many new Azure accounts only have 10 cores available.  If you deploy the Cohesity Cloud Edition for Azure it will need 16 cores per Virtual Machine.  In a production environment I am sure people have high enough quotas to support the three node minimum, but you want to make sure this is set in the regions you are working with.

Create an Azure Service Principal

You are going to use this for multiple things at some point not just to deploy the appliance.  If you want to connect that appliance to your Azure environment to do native backups you will need it as well.  This process is pretty well documented and is easy enough to set up you just need to make sure you have it done and record the keys you create.  The key is to also make sure you assign your app as OWNER to the subscriptions you are going to use it with.

Create A Virtual Network For Cohesity Using The Same Subscription

This is something that will vary by user, but what I found was to create a new Virtual Network that the Deployment Machine and the Cohesity node will both live on.  I also found that the two machines must be contained in the same subscription.  At one point I had the deployment server in a trial, but still using the same virtual network, and things failed.  Instead of going to deep into it I just made sure to put everything on the same virtual network.  Below you can see both the adapters for the two machines in the same virtual network.  Also, a VPN network is NOT needed at this point.  You can do all the setup with public IP addresses, SSH and HTTPS.  Once you connect to Helios you can actually disable all the individual rules if you wanted to later.

Deploy The Cohesity Setup Machine

This machine is already in the Azure catalog.  It is NOT an actual Cohesity node, to the name is a little odd.  When you deploy it be sure to make the user “cohesity” as it will make life easier based on the scripts that you will use.  Simply search for “Cohesity” and you will see it to deploy.

NOTE:  Use whatever settings you want but be sure to attached to the network you created above and set the user to “cohesity”.  If you use a different user you may have to do some extra steps outlined below.

Set up a public IP as well and SSH so you can access the box without any VPN.  You can also use that SSH connection if needed to access SSH on the Cohesity node once it’s deployed and only open HTTPS to that for setup and management.  When you deploy the easiest thing to do is to deploy using the login user of “cohesity” per the deployment guide.  If not you will have to make some other adjustments.

Download The Cohesity VHD and Deployment Tools

This setup you will need access to the Cohesity Downloads page to access.  The links you want are highlighted below.  Be sure to grab the latest ones which currently as of writing this is 6.0a

Pretty simply, connect to the deployment machine and execute the following commands

wget http://<LINK TO VHD DOWNLOAD>
wget http://<LINK TO TOOLS DOWNLOAD> 
tar -xzvf installer-cohesity_azure_setup-<TOOLS VERSION DOWNLOADED>.tar.gz
sudo ./install.sh
mv ./cohesity-<releaseNum>_release-<build>.vhd /home/cohesity/software
cd /home/cohesity/software


####If you did not deploy with cohesity user change password and SU to cohesity

passwd cohesity
su cohesity

#Add $PATH and load profile

echo 'export PATH=$PATH:$HOME/software' >> $HOME/.bash_profile
source .bash_profile
cd /home/cohesity/software

At this point you are ready to start the deployment of Cohesity nodes as the cohesity user.

Setup and Execute Deployment

The first thing you will need to do is edit the params.json file with your settings.  It should look like something below.  I have made a few notes on some of the areas to save you some time.  I will note you cannot use any existing Resource Groups, this will always create a new group.

{
  "azure_application_id" : "APP ID FROM SERVICE PRINCIPAL CREATE EARLIER",
  "azure_application_key" : "APP KEY FROM SERVICE PRINCIPAL CREATE EARLIER",
  "azure_tenant_id" : "YOUR TENNANT / DOMAIN ID",
  "azure_subscription_id" : "YOUR SUBSCRIPTION ID",
  "cohesity_azure_cluster_resource_group_name" : "cohesityce",
  "cohesity_azure_vpn_resource_group_name" : "cohesity",
  "cohesity_azure_vpn_virtual_network_name" : "cohesity-vnet",
  "cohesity_azure_vpn_subnet_name" : "default",
  "cohesity_azure_cluster_location": "centralus",
  "cohesity_azure_domain_name": "FQDN OF THE CLUSTER",
  "cohesity_azure_ntp_servers": "pool.ntp.org",
  "cohesity_azure_dns_server": "8.8.8.8",
  "cohesity_setup_tool_dir_full_path": "/home/cohesity/software",
  "cohesity_setup_templates_dir_full_path": "/home/cohesity/software/conf",
  "cohesity_azure_vhd_file_path": "/home/cohesity/software/cohesity-<releaseNum>_release-<build>.vhd",
  "cohesity_azure_num_vms": 1,
  "cohesity_azure_vm_ip_addresses": "192.168.140.140",
  "cohesity_azure_vm_type": "Standard_DS5_v2",
  "cohesity_azure_num_vms_per_storage_account": 8,
  "cohesity_azure_num_vms_per_resource_group": 64
}

• cohesity_azure_cluster_resource_group_name:  MUST BE lower case and less than 1o characters
• cohesity_azure_cluster_location:  Should match the location of the deployment machine
• cohesity_azure_num_vms:  3 minimum for production deployments
• although network says “VPN” it can be a regular virtual network

One you have set this up you can test the settings.

sudo ./cohesity_azure_setup validate -cohesity_azure_setup_params_file=/home/cohesity/software/params.json

If the test passes you can try to execute it.

NOTE:  If you are using the 6.0a tools you will need to add a $PATH first or the next step will most likely fail at the final cluster creation step.  This is being resolved in the 6.1 tools.  You can add the path temporarily with the command shown below.

sudo ./cohesity_azure_setup create_cluster -cohesity_azure_setup_params_file=/home/cohesity/software/params.json

This process will take about 15 minutes or so and you can watch the progress in your SSH session.  What it is completed you will end up with a Cohesity node, or three.

At this point you can set up an external IP and Network Security Group for HTTPS to map to the node so you can finish the setup as normal without any VPN.  Once you have set up the node you can administer it via HTTPS, or better yet connect it to Helios and remove the External IP and security group as it will no longer be needed!  You can now add backup jobs for your Azure machines using this cloud based cluster, you can use it as a target for Cohesity replication, that would require a VPN, and use it inside Azure to present Scale Out NFS services for SMB shares to your Azure machines.  You could even stand up nodes in different regions and replicate between them.  The flexibility of the Cohesity platform is truly endless and there are so many use cases people can think of I am sure for.

Summary

One thing to point out is you can use Cohesity Cloud Editions even if you don’t have on premises clusters.  If you are a truly cloud first shop and even using multiple clouds there are Cohesity Cloud Editions for AWS and GCP as well.  Now you can work on natively backing up your cloud instances and taking advantage of all that the Cohesity Data Platform can do.

Come see us at Microsoft Ignite and we will be showing a lot of this live in the booth, or drop me a line I’d be happy to show people more!

The post How To Deploy the Cohesity Azure Cloud Edition appeared first on Chris Colotti's Blog.

Once again I was in attendance at VMworld 2018, this year working for Cohesity.  I had the chance to get together with long time friend David Davis from ActualTech Media.  I may have been losing my voice a bit by the time we did the interview, and was possibly rambling.  However, I think some of the cool points came across …hopefully.  I was able to drop the Helios name at least once!  Cohesity is truly a data platform just about everyone can use.

The post My ActualTech Media Cohesity VMworld 2018 Interview appeared first on Chris Colotti's Blog.

The last few weeks I have been putting together a new lab for my team at Cohesity to use for live demos.  Part of the overall design was to allow for VPN connections to the various clouds that Cohesity supports so we can demonstrate real world use cases with Cloud Editions running real-time in each cloud connected back to the on premises lab.  While the overall lab architecture is still coming together the core networking we selected to use was VMware NSX.  With that we deployed a core NSX Gateway that is to act as the central aggregation point for internet access and VPN connections.  What I found out quickly is that connecting an NSX VPN to Azure, GCP, and AWS is not very well documented and each one seemed to be slightly different.  So now that it is all done and working I wanted to quickly document each clouds specific settings to work with the VMware NSX Gateway for IPSEC VPN.  Hopefully this saves other the hours of trial and error and troubleshooting it took me.

In order to try to keep this simple and brief I will cover the main points for each cloud that relates to the NSX Gateway settings.  I won’t go into any firewall rules you may need to pass traffic this is just how to get tunnels up and running.  Each cloud seemed easiest to configure as POLICY based.  I am also assuming you have set up the appropriate VPN gateways and networks per each cloud vendor.  Again this is just the settings specific to NSX once you have all the core items deployed including routing tables.  I have highlighted the items that have minor differences.

NSX Static Route Policy Based VPN Settings:

AWS

NOTE:  What wrapped me up was the Local and Peer subnets.  In the other configurations you will see standard comma separated notations and that does NOT work for AWS.  You will still need to propagate your routing tables

NSX Static Route Policy Based VPN Settings:

GCP

NOTE:  I found that if you are using NAT to your NSX Gateway like I am you MUST set the local IP to the actual Public IP.  You will see errors in the GCP connection logs if you don’t.  Unlike AWS you can use comma separated subnet lists.

NSX Static Route Policy Based VPN Settings:

Azure

NOTE:  This one you must have PFS disabled and you must use AES256 but otherwise it is similar to the GCP setup.

The post How To Configure NSX IPSEC VPN With AWS, Azure, and GCP appeared first on Chris Colotti's Blog.

Some days you wake up and wonder ‘What If..?”  For whatever reason today was that day.  I woke up thinking about what would be different if I did this, or made that decision instead.  I pledged a long time ago to myself never to day “Should’ve, Could’ve, Would’ve”, but as the years go by it’s hard not to wonder.  Some people know I started my IT career back in 1998 when I started working at PC Connection taking catalog orders on the graveyard shift for the holidays.  Fast forward to 2004 I got into the IT department building ESX 2.5 for production on IBM Blade Servers.  I was self taught, motivated, and full of questions.  Eventually I made it to VMware where I spent almost 10 years, the longest single stay at a company in my career.

I decided to depress myself and do a little math from my VMware days when I had Stock Options, RSU’s, and Employe Stock Purchases.  I actually went back and looked up all the grants I had…and I say had because they are long gone.  Most of them cashed out through out the years, but this is the “what if” scenario.  All told the would have added up to about 12,233 shares combined, give or take for taxes, etc.  As of closing bell yesterday had I held on to all those they would have been worth about $2,137,105.10.  Imagine my surprise, the things I could have done with that money.  I could have paid off my house, cars, maybe even retired.  Granted some of that did go into building our new house, and helped buy a car at the time.

So why am I thinking about this today?  Honestly I am not sure.  Occasionally I wonder what’s left as I get older when it comes to work and life, and what is it all for.  Most of us don’t leave a “Legacy”.  I’ve never written books, I have no patents, I’ll never be a CEO.  Much of this I have basically accepted so where is my life going?  What’s the next “thing” and fucking hell, was I stupid all those years ago not thinking all that stock would have been worth something.  Funny thing is people ask me all the time why I left VMware.  To be perfectly honest, I didn’t know where else to go there.  I had also been pretty well left out of new equity by my boss at the time for about my last 3 years there.  The thing is once equity grants stop coming there is no overlap on them and eventually you are back to ground zero with nothing vesting.  So in some cases I figured, what’s the point.  If I am going to start over why not start over somewhere else?

What’s the point to this short post?  None really, other than in the last two or three years I’ve been trying to figure out how to get my family to a point where the monthly household expenses are manageable.  Moving to Tennessee cost us a lot.  Even though the living expenses are less expensive, the build of the house, the things we wanted in it, etc were way over budget.  I kept saying “Go Big or Go Home!” except now as I wonder when the sun will set on my career how will those bills get paid.  Some days you just have to wake up and shake your head and say ‘What the fuck was I thinking?”.  We can’t change the past, all we can do is learn from it and move forward.  That does not mean I will make better decisions, but the future is much shorter than it was 20 years ago when I really started my career.  You think you make the best decisions at the time, and in most cases they probably are.  It’s when you look back and wonder, “what if?”.

The post Could Have, Should Have, Would Have…. appeared first on Chris Colotti's Blog.