Friday, October 18, 2013

Moving Exchange Server 2010 Transaction Logs

An Exchange 2010 server that I've recently inherited has been ill lately. Symptoms include low disk space warnings, Back Pressure, and queued messages on the sendmail smarthost. I treated these symptoms with ad hoc backups to commit the transaction logs to the database and adding some space to the VM's vmdks, but after two rounds of treatment I decided it was time to deal with the root cause.

It didn't take long to find the problem: the VM had two volumes: C: and E:. C: was the OS, and E: was Exchange. All of it. Application and data. And logs. And E: was running out of space because backups weren't being run frequently enough. In other words: root cause was poor planning.

Drill down to the Mailbox Database. 
The operational issues were easy enough to fix, though. So here's what I did, and what you can do if you run into this problem.

The Fix

In the Exchange Management Console (EMC), drill down until you find the Mailbox Database associated with the transaction logs you want to move.

On the right side of the EMC, click "Move database path." A window will appear that let's you specify the location for your database and the transaction logs. In the example to the right, note that I've changed the location of the logs to the F drive that I added to the VM.

Click Move to begin the operation.

NOTE: Moving either the database or the transaction logs will temporarily dismount the information store that you're working with. In other words, don't do this when users are connected; plan for a brief outage.

Why are we doing this?

Most engineers will tell you that separating your database files from your logfiles is a best practice for performance reasons. This is typically true, but keep in mind that if you're running your database or Exchange server as a VM, you need to consider that your VMDKs may be on the same LUN. The performance benefit can't be realized in this configuration. However, there is a compelling reason to move your logs even in this case: let's say that your backups aren't running as often as they should. Without a successful backup, your Exchange transaction logs will never be committed to the database, and they will continue to grow at a sometimes alarming rate. If these logs grow too large, the Exchange Back Pressure feature will kick in and defer receipt of inbound email.

Of course, the solution to this problem is prevention. Plan your Exchange server database and logfile placement with availability and performance in mind. Monitor your backups, drive space, and mail queues. Easy. Basic.

Monday, October 14, 2013

VMware vCenter Converter Standalone Error - PEM_read_bio:no start line

I started my day off with a planning meeting to review a group presentation for the client's new CIO. You know, discussions about how much detail to go into with regard to the infrastructure, do we talk about SLAs, management type stuff. We were making great progress when suddenly, one of our help desk guys was lurking in the doorway with a worried look.

"A server just crashed."

No worries, I said. We'll sort it out. Turns out that one of the hard drives failed, but thankfully the RAID saved us (hey, that should be the name of a blog!). The server came back online, but with one failed drive. The hardware was an old Dell PE 1950 that was scheduled for p2v later this year. So we agreed to p2v that server tonight instead.

I fired up the VMware vCenter Standalone Converter, and made the classic mistake: not running it as administrator. Of course, this mistake isn't apparent until you click Finish after defining the conversion job. No worries, I thought. I'll just run it as admin and knock this out. But... no. Here's the error I got after defining a new conversion job:

[01880 error 'Default'] class Vmacore::Ssl::SSLException(SSL Exception: error:0906D06C:PEM routines:PEM_read_bio:no start line)

I checked that I had the correct permissions on the source server, and I did. Checked DNS settings on the source machine; they were correct. Then remembered that I had installed the converter agent during my first attempt to convert the VM, and did so with different AD credentials. This was the root of the problem.

The Fix

To resolve this problem, manually remove the Converter Agent from the source machine. Then you'll need to go back to the beginning of the conversion task wizard to install the agent again (use the back button so all of your values are not lost). Then run the conversion job and you'll be back in business.