Welcome to
Rob Girard dot com
If you see this text, then your browser doesn't support JavaScript or div id replacement.

Headshot pic Email Me! Pics Archive Links Links Search

ARCHIVE: May, 2008

----- 05.27.2008 --------
[GEEK ALERT] Every once in a while I run into something technical at work that I feel I should share since my searches came up blank. If you're not a geek, you should probably skip to my next post ;) Or enjoy it for what it is :)

Possible Bug in VMWare Virtual Center 2.5 with an ESX 3.5 host
So I ran into a problem at work yesterday with VMWare. I needed to deploy a bunch of 64bit Windows 2003 R2 servers, and the easiest way to do this is to build myself a template, customize it, sysprep it, turn it into a template, and then deploy it. Deploying templates in VMWare is quick and easy and you end up with a fully function server in less time that it would have taken you to even unpack a physical server's box, let alone dispose of the cardboard! So... this is where routine turns into WTF?!?!?.

The initial steps that lead to the destruction of my template (which I had just gotten exactly how I wanted!) were a little unclear, but with some poking and prodding, I was able to find the exact steps that lead to my demise. So, instead of wasting your time with an in-depth report of what went wrong and what I "thought" was the problem, I'll give you a nutshell report and then a more detailed summary of my findings so you can avoid this happening to you, and maybe even be able to fix it before it's too late.

Here's the nutshell: I built my template consisting of an 8GB C: drive. I prefer to do this to save on disk space (no thin provisioning yet), and then add data drives to each VM. This also allows me to move a VM's components to different spindles later on to improve performance (OS drive, pagefile, logging, app data, etc....). I deployed 4 instances of the server: 2 for sharepoint (Dev & Testing), and 2 for SQL (Dev & Testing). After deploying, I realized that I actually wanted larger c: drives in this case to closer mimic what was already in production. So, I started destroying my VM's, and then deployed a new one from the template with 1 "minor" change. Right at the end of the Deploy Template wizard, there is an option of "Edit VM Settings (Experimental)". I had used it in the past with success (or so I thought.... I need to confirm this based on my new findings!). I checked the box, then changed the disk from 8 GB to 30 GB.

Now, all's well and I now have a new VM deploying. Just to recap, 1 template and 1 new VM deploying with the disk size altered. The 4 VM's have been deleted from disk and no trace was left.

Now, just like the first go, there's no need to wait for my VM to finish deploying before starting the others. I right-click the template, deploy new VM from template, answer 10 seconds worth of questions, including the option to edit VM settings (* - EXPERIMENTAL), and click finish. I always wondered why it was called experimental... it "appeared" to work fine. About 20% into creating the VM, Virtual Center through an error at me about a locked file. The file was the VMDK for my template. Strange.

I tried to redeploy, and kept getting the same error. Weird. So, I go into the one that did work and decide to make sure the disk size had changed. Well... it didn't, it was still 8 GB's. And now comes the part where I could go on and on about what I thought it was, but instead, I'll fill in the blanks with my new discoveries that only came to me in the night after sleeping on it. Btw, I have this awesome ability to sleep on problems and wake up with answers :) It makes for a tiring night, but is well worth it!

Using Putty, I ssh'ed into the ESX box. I looked at my new VM and found it's disk was actually 30 GB's. Weird, because windows nor virtual center read it like that. And my partial deployment of the 2nd box was on the disk, something that needed to be cleaned up. So, in Virtual Center, I deleted the partial VM from disk. I then tried to redeploy from template, but this time it's a file not found error! Sure enough, my VMDK is now missing! It's the smaller descriptor file that references the vm_name-flat.vmdk. Where did it go?!?!?

Back to my first deployed VM. Start it... ERROR. What? It can't start! Sure enough, my VMX file for the once-working, newly created VM actually referenced the VMDK in the folder of the template!!! That explains the file locking error.... when my first VM deployed, I had the option to power it on. When it powered up after completing deployment, it locked the vmdk in the template's folder, then VC threw the error my way because it was trying to clone the file to create my 2nd VM.

Now, I was starting to panic because I didn't want to rebuild my template.... but a minor inconvenience panic, not the same kind you get when there's production data balancing in your decisions! But at the same time, you have backups of production, but since all this was just built, I had nothing. And it was all pretty routine stuff, so why did I need it (lol... I surprise myself DAILY!).

So now I was stuck with a broken VM, and a deleted vmdk for the template. I copied the vmdk over from the good one, altered the file to point at the right place, crossed my fingers, then deployed the template again. This time it was working.... atleast until it hit around 20% and decided the disk was corrupt. Crap!

So I deleted the template. Altered my VMX file of my semi-good VM, and it was able to boot off its own VMDK. Yeah! But now I had a new problem... it was a 30 GB disk with an 8 GB boot partition cut out and 22 GB's of unpartitioned space. I needed this to be 8 GB, after all, this was my new "golden image" template. After a few ideas, I used VMWare converter to clone it and change the disk back to 8 GB, then converted that into a template and Voila! I had a new template. First I sysprepped it, then converted it back to a template. I was back to where I wanted to be.

Summary and findings:

  • 1. By using the "Edit VM Settings (experimental)" option upon deploying a VM, the VMX file of the VM being deployed reference the VMDK of the template.... not it's own VMDK (which does exist in the same folder as the VM).
  • 2. Not checking this option allows the VM to use the VMDK in its own dir, as intended.
  • 3. Deleting a new VM that is a clone of the template AND checked the "edit VM settings" option WILL result in your VMDK file of the template being deleted from its folder.
  • 4. This occurs on 64bit VM's for sure, but may or may not affect 32bit hosts.... I have not tested this yet.
  • Suggestion/Workaround:
    To be on the safe side, simply deploy the VM from template as usual and do not check the "edit settings" option. If you need to create a larger disk, you have a few options:

  • 1. Use VMWare converter
  • 2. Shut down the VM and use "Edit Settings" in VC to grow the disk, and then mount it to another VM (so it is NOT the boot partition) and run "diskpart.exe" from windows to expand the partition to consume the entire disk.
  • 3. Use vmkfstools to grow the disk, and then mount it to another VM (so it is NOT the boot partition) and run "diskpart.exe" from windows to expand the partition to consume the entire disk.
  • As far as I can tell, this is a bug in VMWare's Virtual Center (v2.5 in my case). I have not been able to find any information on the subject. I hope that if you found this post via search engine that it was useful to you. If you have any further info, I'd love to hear from you.... please email me. Thanks!

    ----- 05.27.2008 --------
    As if last week's tie in soccer wasn't sweet enough, tonight we won! Our first win, but our second game in our non-loss streak! What may have been best is the fact that we went into it sure were going to get crushed. Why? Because they had beaten the team that stomped on us 9-0 a few weeks back. But every week we get better and this week we went out there are fierce as possible. Even when they were beating us 3-0, we felt proud that it wasn't the same crushing we recieved a few weeks ago. And we kept pushing harder and harder, and the next thing you know, we won 4-3! Talk about an underdog story! AWESOME! I am so proud of my team! :)

    Outside of soccer, there's some things in the making, but it's too early to say for sure where they are leading. I guess this is open to interpretation.... you know who you are ;) Anyway, busy day @ work tomorrow and it's getting late. Bed time! Hope you're all doing great! Later!

    ----- 05.21.2008 --------
    Here's a really quick highlight of the past week or so. Soccer last night: We tied! That's right... our first non-loss game :) Score was 1-1.

    Long Weekend: Was awesome! Went down to see my family in Windsor and celebrate my sister Laura's birthday. Nutshell of the trip: Took lots of pics (stay tuned for the posts), roller blading (first tick on the blade-o-meter!), sailing (first time!), playing in trees with a chainsaw, lighting off fireworks, foosball, rock band.... and I'm sure there's more.

    OK, so that's my idea of a really quick post. G'nite... and stay tuned for some pics... hopefully by the end of the weekend, if not sooner.

    ----- 05.13.2008 --------
    Tonight I just got back from a really late soccer match. I'm playing for my previous company and we're awesome! No, we don't score much, and no, we don't necessarily show it on the field yet.... but we're still awesome! We're all rookies and we're out there giving it our all in a Recreational League. And for a bunch of newbz, we're really coming along great! We lost 3-1 tonight... and it didn't feel like a masacre! Last week was 9-0, and the week before 6-1. Tonight's game was at 11PM, which kinda sucked, but we still made it out in decent numbers and gave it our all. I look forward to more games... Tuesday nights are soccer nights!

    So what else is new since my last post? Yikes... that was a long time ago! Well, last week (Monday) I had another follow-up assessment at the gym. I've very happy that I improved more, across every single criteria! Lost pounds and inches, increased strength, lowered heart rate (both resting & active), shaved over 2 minutes off my mile & a half run... and I even increased my flexibility by 4 inches! All this in 6 weeks! :) So after the assessment, I was given a new workout routine... and it's a douzie! A good one though, and one that's tough enough to make my next assessment just as rewarding :)

    What else? Saturday I saw the new Iron Man movie... wicked! Sure, I had to suspend my beliefs to enjoy it, but whilst doing so, it made for a fun flick! After that, I got home and spent all night putting together a BBQ in the dark up on my roof top patio. Totally worth it... Sunday night's steaks were awesome! And I had BBQ chicken last night! (and tonight for leftovers).

    Anyway, all's going well and I figured I'd post while I had the time/energy. Actually, it's really late, but I'm trying to unwind after our intense soccer match tonight. Hope you're all doing great! If I haven't heard from you in a while... drop me a line! This weekend I'm off to Windsor for the long weekend and will be celebrating my sister Laura's birthday. Looking foward to it! I can't forget to bring my Devil sticks.... the weather's great and I need to take my fire skills to a new level :) Stay tuned for the vids & pics. OK... g'nite & cheers to all of you.

    ----- 05.04.2008 --------
    UPDATE: Right after I finished writing my most recent post, I decided to tackle my pics & video from the Don River today, whip something up and post it....sooner rather than later. Enjoy!

    Happy Luke Skywalker Day to everyone! "May the 4th be with you...." :) Yes, geeky.... but that doesn't stop me from using it year after year :) What a weekend! I'm not even sure where to start. I guess it started at the end of last week, finishing off my Solaris 10 admin class. It was really enjoyable, but intense. Which brings me to Friday night... a fun night of a few (dozen) brews at Victory Cafe with some friends of friends. Cool peeps that I hope to party again with sometime in the near future.

    Then Saturday came rolling along and my sister Laura and her friend Caroline came up for a visit. During the day we did some shopping at MEC and EuropeBound where I mainly studied maps and got excited about the Algonquin voyage at the end of July. Later on Saturday, we headed out to my Cousin Scott's in Scarborough for his annual BBQ & daughter's birthday. Then we headed back to the city to make it not too late a night. And why did it matter, because this morning I needed to be up bright and early.

    And this takes me to what is most likely the highlight of the weekend... I Paddled the Don River with some of my coworkers. We started up at Leslie and Eglington and paddled our way down to the lake. There was some good current surging through the river, varying from slight riffles in the water to small rapids. Small... but not to be underestimated! Funny enough, my canoe partner Steve & I capsized! We rounded a corner and the river was tight and fast moving. Right infront of us was a "strainer": low hanging tree branches that were just barely hanging above the water's surface. In an effort to duck them, the canoe went over and we got strained by the strainer. Hilarious! And even better, I've got the "water-proof" video to go with it! Amazingly, when it happened, my camera turned on and started recording while underwater. It's really quite funny and once I get a chance to edit it out a bit, I'll post it for you to see. Stay tuned...

    All in all, a great weekend! Even with the dip in the cold water, it was exciting and hilarious and definately makes for a better story than a dry trip down a river. In all my years of canoeing, this was my first (accidental) tip! LOL! Even in the class II & III rapids on the mighty Missinaibi, we swamped, but didn't go over. A first time for everything!

    Anyway, I anticipate a big day at work tomorrow since I was on course all last week, so I'm going to stop here and hit the sack nice and early tonight. I've been up since 6 and today's been a LONG day. And I got my taxes done today too :) (Where did that come from?!?!). OK... g'nite!


    Search WWW Search robgirard.com

    © 2000, Rob Girard dot com