VMWare
I remember my first ESX cluster. It was a game changer! In my career, up until that moment almost a decade and a half ago, servers, in my (and most other IT pros) were physical. Or Bare Metal if you will, but that was not a thing back then. It has been so amusing to think back on how times have changed, and I find myself giggling a little while reminiscing!
Physical servers.
Or just servers.
That was the standard.
Virtual Servers had recently gone mainstream with the VMWare product line but was at this time still met with skepticism.
I was one of those skeptics.
Virtual Server was mainly a prototype concept back then, and no one thought of them as production environments. Sure, it could be useful in some lab scenarios, but there is no way I’ll put production applications in there!
Just think of the performance loss!
Think of the risk!
Think of the children!
Also, If I go virtual, what do I do with these 100 square meters of server boxes, racked up in these 50 cabinets, consuming the power of a medium sized city? These boxes, that has been running between 1-10 years, and to whom I’ve given individual names (the naming theme was Nordic mythology first, but the names ran out so I continued with car brands, but they’ll also run out soon, so I’ll have to pick a new theme, I’m leaning towards rock bands), and where I know each and every one of these servers like the back of my hand? Where I have nurtured and tweaked and upgraded them caringly and got them working just the way I want? Where I whisper sweet words to them, and sometimes scream at them and most of the time beg them not to freeze up and die on me? Where we have an understanding for each other’s needs?
My family?
Ok, sometimes they won’t do as I say and crash on me and refuse to reboot.
And sometimes, when the power spikes and the main shuts down, I must start them in just the right order to not flip the main again. And only I know that order, it’s a secret between me and my servers.
My precious.
And yes, after a power loss, usually only 80 percent ever start again.
But that’s ok.
I have another 100 square meters of space over there, where I store a decade worth of old components and spare parts. I never throw anything away, what are you crazy?
You need spares for all components, in all servers ever put into production! At least two of every component! And usually it’s best to have a cold standby of every server box. I always buy two when I need one new server! And you never know when some old interface comes back into fashion again, so let’s save every piece of kit ever made! Token ring might be a thing again, so let’s save these perfectly fine couple of dozen switches for another decade! Remember when I threw away that old SCSI1 card and was unable to connect that hard drive where I had made a backup of the CEOs porn stash kid’s pictures? He was furious and it took me three weeks to find another card and hook it up in a machine and get it recognized by the OS, just to discover the hard drive was dead.
And even if a server did die on me permanently, that’s no problem at all!
See, over here I have yet another 50 square meters packed with cabinets full of backup servers and DLT robotic libraries! State-of-the-art! I don’t even have to switch the backup tapes in the middle of the night anymore! It’s a robot that does it for me! And it only fails less than 50% of the time! And the only maintenance needed is to empty all the tapes from the trays, clean them from dust dressed in that protective suit over there and put in new tapes once a day! It only takes around 3 hours! And I have fresh backups of everything, only a couple of days old copies since the backup jobs only take around 48 hours to complete! At least 50% of the times at least, since you never know what has failed. Not until you try to restore, and usually the backup files and/or media is corrupt unfortunately. But that’s just the way it is with back-up!
It’s a state-of-the-art implementation!
And for compliance reasons I store all the latest tapes in this huge safe over here! We had to tear down and rebuild tree walls to get it in! We’ll never be able to move out of these offices! And, also for compliance, I send a copy of last months’ backups to a super secure backup storage facility deep in the mountains in a secret location ready for swift retrieval within 10-15 business days.
You see, a total server crash is no problem for me!
Even if the server doesn’t start after I’ve replaced every single component in it, test booting it after each and every component swap, I just rack up a new server, fit memory, hard drive, peripherals, power supply, cords and cables, install the OS, drivers and applications using these 5 floppy disks over there, and 19 CDs over here! Ok usually at least one of the install media is dead and I have to make a new copy, but that’s no problem because I have this ultra-fast CD-burner over there where you can burn a new cd in less than 3 hours! And then, when you have the server up and humming, you just have to restore the data from backup! Usually that works after having gone through some handful of backups, locating one where the tape is not dead, and the files are not corrupt. And sometimes, you only lose like a weeks’ worth of data!
Boom!
4 days of working round the clock, and the server is restored!
Like if the crash had never happened!
By the way, see that sweet couch in the back of the server room? That’s where I sleep. I found it’s just easier to leave the server room as little as possible. The family needs me by its side, round the clock. And see that micro oven over here? That’s where I cook my meals. And the builders are going to install a toilet over here when we re-do the office next time, because now I pee in this cup and that’s a risk, having liquids so close to the servers. Shower? No, that’s not a priority.
Sorry, what did you say? My tinnitus is getting really bad after having spent years in here.
Do I live in the server room?
No, not at all, I have a flat a ten-minute walk from the office. Well I think it’s still there at least. I haven’t been there for a while, and I think I left it unlocked so maybe someone else lives there now. And last time I needed to go there, I couldn’t find the way since my eyes no longer work in daylight, and my skin gets badly burnt after 10 seconds in the sun.
But let me tell you! We have a good life here in the server room!
Me and my family!
My precious.
And let me show you this state-of-the art halon fire extinguisher system! And this state-of-the-art UPS unit covering that wall over there! And the state-of-the-art backup-power diesel generator down in the basement! And this state-of-the-art anti-static, flood-safe double flooring! Or this state-of-the-art…
Ok, I may be exaggerating a little, but tending physical servers certainly comes with its own set of problems. And characters.
Virtual Servers
15 years ago, virtual machines were not a new thing. On Unix it had existed for years. But for Windows, VMWare changed the landscape completely!
I had tried running some servers on VMWare Server, the free server version, and I thought of it as slow and not very trustworthy. But I started to cram more and more servers on there, mainly development stuff and stuff I didn’t need backup of. I could fit around 8 VMs per host.
And when no rack space was left, and I had grown very tired of the developers cramming all instances onto a single VM, I decided to buy my first ESX cluster, complete with hosts, a brand-new SAN and EMC storage, top-of-the-line Dell servers sporting the obscene amount of 32 GB RAM each!
I could not believe it!
I would convert a physical server within hours and just cram it in there!
In a couple of months, I had virtualized every single server!
I had turned from a doubter into a believer into a missionary, preaching the god word of virtualization.
I became a true “VM-head”!
And I was not alone. ESX/vSphere became the de-facto standard in all server rooms around the world and VMWare reached total market domination.
The last couple of years, that wonderful, warm, tingling feeling I’ve had for VMware has faded.
VMware’s server offerings and pricing has spiraled out of control, rebranding stuff weekly, separating and consolidating services in different packaging, trying to re-invent itself through bullshit marketing and buzzword bonanzas. A lot of crap, such as vRealize and Lab Manager, has been flaunted and the only truly great product, vSphere, has been hidden far back in the catalogue.
When some idiot decided to depreciate the vSphere C# client in favor of the fucking abomination web-based client built in adobe flash(!), the bubble burst and I no longer felt admiration for them.
The cloud
VMWare, of course has lost relevance now that the cloud is totally mainstream, and the first choice for running any new workload being developed and/or deployed. Sure, VMWare tries to stay relevant and sneak back in the picture frame with the cool kids, pitching vSphere on AWS and what not.
But to me, running a VM (or container on top of a VM) on vSphere on top of AWS is, best case, a temporary solution during migration to the real cloud, and worst-case, idiocy. To me, someone running a hypervisor on top of cloud fabric, itself powered by a hypervisor seems unlikely to exist in a couple of years. But maybe VMware will flourish again if they are released from the clutches of Dell. Maybe they’ll get their own Nadella who can turn it around. But to me they’re legacy. And I predict they will go down. I just hope they don’t drag a generation of poor, server room-dwelling admins with them.
P.S
I often think of my dear former colleague Peters words of wisdom regarding VMs: “We (the developers and Ops) treat them as pets when they should be treated as cattle!” This is my take on that quote: When a server is headless, we’ll treat it like cattle When it’s virtual we treat it like a pet When it’s physical we treat it like our first-born child
How will we think about our servers in the future? Hopefully not at all.
The personal experiences, viewpoints and opinions expressed in this blog post are my own and in no way represent those of the company.