Understanding Active Manager in Exchange 2010 Server

Microsoft Exchange Server 2010 includes a new component called Active Manager that provides functionality that replaces the resource model and failover management features provided by integration with the Cluster service in previous versions of Exchange. Exchange no longer uses the cluster resource model for high availability. All Exchange cluster resources provided by exres.dll no longer exist, including the construct known as a clustered mailbox server. A Windows Failover Cluster is used by Exchange, but there are no cluster groups for Exchange, and there are no storage resources in the cluster. Thus, if you examine the cluster using cluster management tools, you’ll see only the core cluster resources (IP Address and Network Name, and if needed, quorum resource). Cluster nodes and networks will also exist, but those are managed by Exchange and not cluster or cluster tools.

Active Manager runs as a role on all Mailbox servers. On Mailbox servers that are not configured for high availability, there is a single Active Manager role: Standalone Active Manager. On servers that are members of a database availability group (DAG), there are two Active Manager roles: Primary Active Manager (PAM) and Standby Active Manager (SAM). PAM is the Active Manager in a DAG that decides which copies will be active and passive. PAM is responsible for getting topology change notifications and reacting to server failures. The DAG member that holds the PAM role is always the member that currently owns the cluster quorum resource (default cluster group). If the server that owns the cluster quorum resource fails, the PAM role automatically moves to a surviving server that takes ownership of the cluster quorum resource. In addition, if you need to take the server that hosts the cluster quorum resource offline for maintenance or an upgrade, you must first move the PAM to another server in the DAG. The PAM controls all movement of the active designations between a database’s copies (only one copy can be active at any specified time, and that copy may be mounted or dismounted). The PAM also performs the functions of the SAM role on the local system (detecting local database and local Information Store failures).

The SAM provides information on which server hosts the active copy of a mailbox database to other components of Exchange that are running an Active Manager client component (for example, RPC Client Access service or Hub Transport server). The SAM detects failures of local databases and the local Information Store. It reacts to failures by asking the PAM to initiate a failover (if the database is replicated). A SAM doesn’t determine the target of failover, nor does it update a database’s location state in the PAM. It will access the active database copy location state to answer queries for the active copy of the database that it receives.

 

How to do Bulk E-Mail Filtering in FOPE

Forefront Online Protection for Exchange (FOPE) identifies inbound bulk mail (such as advertisements and marketing emails) by marking a stamp in the message headers. FOPE inserts the X-Forefront-Antispam-Report header into each message it scans. If a message is identified as a bulk mail message, FOPE inserts SRV:BULK into that header.

Administrators can create a rule on their mail server (such as Exchange Server 2007 or 2010) that moves all unwanted mail for all their users to the Junk Mail folder based upon this stamp in the message headers. For more information, see http://social.technet.microsoft.com/wiki/contents/articles/2922.bulk-mail-filtering-in-fope.aspx#_Blocking_Unwanted_Bulk

Users can create a rule in their local email client (such as Microsoft Outlook) that moves unwanted mail to their Junk Mail folder based upon this stamp in the message headers. For more information, see http://social.technet.microsoft.com/wiki/contents/articles/2922.bulk-mail-filtering-in-fope.aspx#_Blocking_Unwanted_Bulk_1

Blocking Unwanted Bulk Emails Using Exchange Transport Rules

To perform the following procedure on a computer that has the Edge Transport server role installed, the account you use must be delegated with the Exchange Organization Administrator role. Additionally, you must log on by using an account that is a member of the local Administrators group on that computer. For more information about permissions, delegating roles, and the rights that are required to administer Exchange Server, see Permission Considerations .

The following procedure shows you how to open the Transport Rule wizard on a Hub Transport server or an Edge Transport server in the Exchange Management Console, and then create a new transport rule that will block unwanted bulk emails. Once the wizard is open, the steps are the same for both roles.

To use the Exchange Management Console to create a new transport rule

1. Open the Exchange Management Console on the Hub Transport server or the Edge Transport server on which you want to create the new transport rule.

2. To open the Exchange Transport Rule wizard in the Exchange Management Console, in the console tree, click Organization Configuration, and then click either Hub Transport or Edge Transport. In the resulting pane, click the Transport Rules tab, and then in the Actions pane, click New Transport Rule.

3. In the Name field of the New Transport Rule wizard, type the name of the transport rule: Bulk Mail Filtering.

4. If you have notes for this rule, in the Comments field, type the notes.

5. If you want the rule to be created in a disabled state, clear the Enabled check box. Otherwise, leave the Enabled check box selected, and then click Next.

6. In the Step 1. Select conditions box, select the when the message header contains specific words check box.

7. In the Step 2. Edit the rule description by clicking an underlined value box, click the message header link.

8. In the Specify message header dialog box, in the Message header field, type X-Forefront-Antispam-Report, and click OK to close the window.

9. In the Step 2. Edit the rule description by clicking an underlined value box, click the specific words link.

  1. In the Specify words dialog box, in the text box, type SRV:BULK, click Add, click OK to close the window, and then click Next.
  2. In the Step 1. Select actions box, select the set the spam confidence level to value check box.
  3. In the Step 2. Edit the rule description by clicking an underlined value box, click the 0 link associated with this item, and then set the Spam confidence level (SCL) threshold to 6. This moves all identified bulk email to your Junk Mail folder. Click OK and then click Next.
  4. In the Step 1. Select exceptions if necessary box, select all the exceptions that you want to be applied to this rule. You are not required to select any exceptions.
  5. If you selected exceptions in the previous step, in the Step 2. Edit the rule description by clicking an underlined value box, click each link.
  6. When you click a link, a new dialog box opens to prompt you to select the items that you want to add, or to type the values manually. When you have finished, click OK to close the window.
  7. Repeat the previous step for each exception that you selected. After you configure all the exceptions, click Next.
  8. Review the Configuration Summary. If you agree with the configuration of the new rule, click New, and then click Finish.

 

Blocking Unwanted Bulk Emails Using Outlook Rules

To create an individual message rule in Outlook to filter bulk email and move it to your Junk Mail folder, perform the following steps.

1. In Outlook 2003 and Outlook 2007:

a) In the Navigation Pane, click Mail.

b) On the Tools menu, click Rules and Alerts.

c) If you have more than one email account, in the Apply changes to this folder list, click the Inbox you want.

In Outlook 2010:

a) Under the Rules drop-down menu, click Manage Rules and Alerts.

2. Click New Rule.

3. Under Start from a blank rule, click Apply rule on messages I receive, and then click Next.

4. In the Step 1: Select conditions box, select the with specific words in the message header check box.

5. In the Step 2: Edit the rule description box, click the specific words link.

6. In the subsequent dialog box, type SRV:BULK in the text box, click Add, click OK, and then click Next.

7. In the Step 1: Select actions box, select the move it to the specified folder check box.

8. In the Step 2: Edit the rule description box, click the specified link.

9. Select the Junk E-mail folder, click OK, and then click Next.

  1. Specify any exceptions, if desired. For example, you can add a sender to your Contacts list and update the rule to not move messages from your Contacts to your Junk Mail folder:

a) For the exception, select the except if sender is in specified Address Book check box and then click the specified link in the rule description area.
b) Under the Address lists, click Contacts, click Add, and then click Next.
Note: Going forward, if you want to receive email from a particular sender, you must add them to your Contacts list. Adding them to your Safe Senders list will not prevent the Outlook rule from moving it to your Junk Mail folder.

  1. Under the name for the rule, specify Bulk Mail Filtering and then review the rest of your settings. By default, Turn on this rule is selected. Optionally, if you want to run this rule on messages that are already in your Inbox, select the Run this rule now on messages already in “Inbox” check box. When you are done with your selections, click Finish.

Single Item Recovery in Exchange Server 2010

Introduction

Users often delete data from their mailboxes that they later want recovered. Recovery of these deleted items is the most common reason for IT admins to recover data from traditional point-in-time backups today.

With previous versions of Exchange Server, administrators implemented two solutions to enable single item recovery, dumpster and restores. Both had their issues, unfortunately.

Exchange 2010 aims to reduce the complexity and administrative costs associated with single item recovery.

Terminology

The following definitions may be useful for understanding the content within this article:

  • Store Soft Delete – this is when an item has been deleted from the Deleted Items folder and placed within dumpster.
  • Store Hard Delete – This is when an item has been marked for purging out of the store.
  • Hard Delete via Outlook (Shift-Delete) – This is when the end user performs a shift-delete on an item. This results in the item bypassing the deleted items folder and being directly placed in dumpster.
Single Item Recovery in Previous Versions of Exchange

The end user single item recovery functionality was enabled through the store via the store dumpster. Administrators configured the dumpster setting on a per database or per mailbox basis. The default deleted item retention window in Exchange 2003 is 7 days, while with Exchange 2007 the default is 14 days.

The end user recovery process worked typically like this (see figure 1):

  1. User receives message.
  2. User deletes the message. The message is moved to the Deleted Items folder.
  3. The user empties the deleted items folder (or an automated process like Managed Folders or Records Management deletes items out of the deleted items folder on a scheduled basis).
  4. The user then realizes he requires the previously deleted item. Using Outlook the user accesses Recover Deleted Items.
  5. Assuming the deletion timestamp of the deleted item is still within the deleted item retention period, the user has two choices:
    1. The user can recover the item. The recovered item is placed back in the Deleted Items folder.
    2. The user can purge the item from recover deleted items. This results in the deletion of the message permanently from the user’s mailbox.

Figure 1. Dumpster in Previous Versions

If the item’s deletion timestamp is beyond the deleted item retention period, then the item is not available for end user recovery. Instead, the user must call Help Desk and request recovery of the item. This involves:

  1. The user knowing when he deleted the item.
  2. The Exchange administrator locates a backup that contains the item in question.
  3. The Exchange administrator successfully restores the database to a recovery storage group.
  4. The Exchange Administrator extracts the deleted item and provides it back to the user. This of course assumes that there is a policy that allows for single item restoration and that this process isn’t a really low priority for the Exchange administrator.

If the backup is invalid, or if there is no backup for the time period in question, then the deleted data is unrecoverable.

The other issue that needs highlighting is the fact that in previous versions of Exchange there is no way to prevent the end user from purging the data from the Recover Deleted Items view. This poses a significant legal risk for organizations that must ensure compliance requirements are met.

Dumpster Changes

In previous releases of Exchange Server, dumpster is essentially a view stored per folder. Items in the dumpster (henceforth known as Dumpster 1.0) stay in the folder where they were soft-deleted (shift-delete or delete from Deleted Items) and are stamped with the ptagDeletedOnFlag flag. These items are special-cased in the store to be excluded from normal Outlook views and quotas. In addition, data with this flag cannot be searched or indexed.

Note: Users can perform a shift-delete and cause a message to bypass the Deleted Items folder and go straight to dumpster. Prior to Outlook 2007, the Recover Deleted Items tool, by default, only exposed the dumpster view for the Deleted Items folder. By setting the DumpsterAlwaysOn registry key (http://support.microsoft.com/kb/886205) you can enable the Recover Deleted Items tool for all folders in the mailbox and thus expose dumpster data for any folder in the Exchange 2003 or Exchange 2007 mailbox.

One of the key architectural changes in Exchange 2010 is to truly enable a litigation hold experience for customers that have legal compliance requirements. As a result, Exchange 2010 must meet these requirements:

  • Exchange has to ensure that dumpster data moves with the mailbox.
  • Dumpster data must be indexed and discoverable.
  • Dumpster must have a quota.
  • Exchange has to prevent purging of data from dumpster.
  • Exchange has to track edits of certain content.
  • Dumpster should be per mailbox and not per folder.

In order to facilitate these requirements, Dumpster was re-architected. Unlike Dumpster 1.0, Dumpster 2.0 is no longer simply a view. Dumpster in Exchange 2010 is implemented as a folder called the Recoverable Items and is located within the Non-IPM subtree of the user’s mailbox (note that this is a hidden section of the mailbox and is not exposed to the end user through any client interface). The folder has three sub-folders:

  1. Deletions
  2. Versions
  3. Purges

The Deletions folder replaces the ptagDeletedOnFlag view that was displayed when a user accessed the Recover Deleted Items tool. When a user soft deletes or performs an Outlook hard delete against an item, the item is moved to the Recoverable ItemsDeletions folder. When the user accesses Outlook/OWA Recover Deleted Items, the RPC Client Access service translates the request and returns the Recoverable ItemsDeletions folder view.

The Versions and Purges folders will be covered in the Single Item Recovery in Exchange 2010 section.

By architecting Dumpster to be a folder, three of the requirements are immediately met:

  • Dumpster data is now indexed and discoverable.
  • Dumpster data can now be moved with the mailbox.
  • Dumpster data is now stored on a per-mailbox basis rather than a per folder basis. From an end-user perspective, this means that deleted data is now easier to recover because Recover Deleted Items tool will now expose deleted data across the entire mailbox.

To ensure that Denial of Service attacks by placing large quantities of data into dumpster are prevented, Dumpster 2.0 has configurable quota settings. These settings can be configured per database and per mailbox:

  • RecoverableItemsWarningQuota – Sets the soft limit that defaults to 20 GB. When the Recoverable Items folder reaches that size, the Exchange administrator will be notified via an event log alert. This alert should fire at the time the mailbox reaches the limit and once daily afterward. By default, the mailbox is not configured with this property, meaning that database-level limits are used. At this limit items will begin to be deleted from the dumpster using the first-in / first-out (FIFO) principle – essentially, the oldest items in the dumpster are deleted first. For example, consider a mailbox that has a dumpster that is 20GB in size and the user deletes an additional 50MB worth of data. In this case, the oldest 50MB worth of data is deleted to make room for the new 50MB worth of data.
  • RecoverableItemsQuota – Sets the hard limit that defaults to 30 GB. At that limit, soft deletes will fail. The Exchange administrator will be notified via an event log alert. This alert should fire at the time the mailbox reaches the limit and once daily afterward. By default, the mailbox is not configured with this property, meaning that database-level limits are used.

Note: Exchange 2010 includes capability for each mailbox to also maintain an archive mailbox as well. There is a dumpster for both the primary mailbox and the archive mailbox. Data deleted in the primary mailbox is placed in the primary mailbox dumpster, while data deleted in the archive mailbox is placed in the archive mailbox dumpster.

Single Item Recovery in Exchange 2010

But how does Exchange 2010 meet the other two requirements, ensuring data is not either accidentally or maliciously purged and that versions are tracked? Exchange 2010 now includes two mechanisms to meet those requirements:

  1. Short-term preservation of data
  2. Long-term preservation of data
Short-Term Preservation of Data

Exchange 2010 includes the ability to ensure that data within the mailbox is preserved for a period of time. This feature can be enabled enabled on a per mailbox basis by running the following cmdlet:

Set-Mailbox -SingleItemRecoveryEnabled $true

Note: When enabling this feature, you will be notified that it could take up to 60 minutes for single item recovery to take effect. This is an approximation due to Active Directory replication. Be sure to evaluate your Active Directory replication topology with respect to administering recipients to understanding how Active Directory replication may impact changes in your environment.

The time period by which the deleted data is maintained is based on the deleted item retention window. The default time period is 14 days in Exchange 2010 and is configurable per database or per mailbox. The following cmdlets let you alter this behavior:

For the mailbox database: Set-MailboxDatabase -DeletedItemRetention

For the mailbox: Set-Mailbox -RetainDeletedItemsFor

Note: Regardless, of whether you have Single Item Recovery enabled, calendar items are maintained in the Recoverable Items folder structure for 120 days. Long-term data preservation via litigation hold will disable the expiration of the items.

At this point you may be thinking, how is this any different than in previous versions of Exchange? With short-term preservation deleted items will still be moved into the Recoverable Items folder structure. However, the data cannot be purged until deletion timestamp is past the deleted item retention window. Even if the end user attempts to purge the data, the data is retained. Consider this example by a malicious user:

  1. User sends or receives a message that is legally incriminating.
  2. User deletes the message. The message is moved to the Deleted Items folder.
  3. The user empties the deleted items folder.
  4. The user accesses the Recover Deleted Items functionality in Outlook or OWA.
  5. The user then selects the item and deletes the item. At this point the user believes he has removed the incriminating evidence. And this is a key strength in this work flow as the end user’s actions are not interrupted or prevented; in other words, the end user’s work flow is not impaired with single item recovery enabled.

However, the message was not purged from the mailbox store. Instead the message was moved from the Recoverable ItemsDeletions folder to the Recoverable ItemsPurges folder. All store hard-deleted items end up in this folder when single item recovery is enabled. The Recoverable ItemsPurges folder is not visible to the end user, meaning that they do not see data retained in this folder in the Recover Deleted Items tool.

When the message deletion timestamp has exceeded the deleted item retention window, Records Management will purge the item. See Figure 2 for a visual representation of this behavior.

Figure 2. Dumpster 2.0

Not only does short term preservation prevent purging of data before the deleted item retention window has expired, but it also enables versioning functionality. Essentially when an item is changed, a copy-on-write is performed to preserve the original version of the item. The original item is placed in the Recoverable ItemsVersions folder. This folder is not exposed to the end user. What triggers a copy-on-write?

  • For messages and posts (IPM.Note* and IPM.Post*), copy-on-write will capture changes in the subject, body, attachments, senders/recipients, and sent/received dates.
  • For other types of items, copy-on-write will occur for any change to the item except for moves between folders and read/unread status changes.
  • Drafts will be exempt from copy-on-write to prevent excessive copies when drafts are auto-saved.

The data stored in the Recoverable ItemsVersions folder is indexed and discoverable by compliance officers.

Long-Term Preservation of Data

Customers sometimes require mechanisms by which data is maintained for longer periods of time, say indefinitely. This is typically due to litigation hold that occurs when the organization is undergoing a lawsuit. With Exchange 2010, litigation hold can be enabled via the Exchange Control Panel or by setting the property LitigationHoldEnabled via the Set-Mailbox cmdlet.

Note: When enabling this feature, you will be notified that it could take up to 60 minutes for single item recovery to take effect. This is an approximation due to Active Directory replication. Be sure to evaluate your Active Directory replication topology with respect to administering recipients to understanding how Active Directory replication may impact changes in your environment.

When litigation hold is enabled, records management purging of dumpster data ceases. Consider the following four cases:

  1. When the end user deletes an item from the “Deleted Items” folder or shift-deletes Outlook/OWA from any folder (soft delete), the item is moved to Recoverable ItemsDeletions folder, where it can be recovered in the Outlook /OWA “Recover Deleted Items” view.
  2. If the end user purges data from the “Recover Deleted Items” view (hard delete from the Recoverable ItemsDeletions folder), the item will be moved to the Recoverable ItemsPurges folder.
  3. When an end user modifies an item, a copy of the original item is placed in the Recoverable ItemsVersions folder based on the criteria discussed in the Short-Term Preservation of Data section.

Also, when litigation hold is enabled, the FIFO deletion at warning limit is ignored . When a user’s Recoverable Items folder exceeds the warning quota for recoverable items (as specified by the RecoverableItemsWarningQuota parameter), an event is logged in the Application event log of the Mailbox server. When the folder exceeds the quota for recoverable items (as specified by the RecoverableItemsQuota parameter), users won’t be able to empty the Deleted Items folder or permanently delete mailbox items. Also copy-on-write won’t be able to create copies of modified items. Therefore, it’s critical that you monitor the Recoverable Items quotas for mailbox users placed on litigation hold.

Recovering Purged Data

Data that is stored in the Recoverable ItemsPurges folder is not accessible or discoverable by the end user. However the data is indexed and discoverable for those that have the proper access role in the Exchange organization role. Role Based Access Control (RBAC) provides the Discovery Management role to allow secure search access to non-technical personnel, without providing elevated privileges to make any operational changes to Exchange Server configuration. Compliance officers or other Exchange administrators with the Discovery Management role can leverage the Exchange Control Panel (ECP) to perform discovery searches using an easy-to-use search interface.

When a single item recovery request is received by help desk, the following actions can be taken:

  1. Either Help Desk or a member of the Discovery Management role utilizes the Exchange Control Panel to target a search against the user’s mailbox to locate the data in question. The framework to perform the search allows the administrator to use Advanced Query Syntax. The recovered data is then extracted and placed in the Discovery Mailbox in a folder that bears the user’s name and the date/time that the search was performed.
  2. The administrator then opens the Discovery Mailbox via OWA or Outlook and verifies that the content recovered is the content requested by the end user.
  3. The administrator then leverages the Exchange Management Shell to perform an export-mailbox against the discovery mailbox targeting the end user’s mailbox. The data is then placed back into the end user’s mailbox.
  4. Help Desk can close the ticket.
Backups and Single Item Recovery

Naturally after understanding the features included in Exchange 2010, a logical follow up question is “Do I still need backups for single item recovery?” The answer depends on your backup requirements and your capacity planning.

Today many customers minimize the deleted item retention window, yet they maintain long backup retention time periods (from 14 days to several months to years).

Let’s consider a customer that currently maintains backups for 90 days and only retains deleted items within Exchange for 5 days. This customer is performing backup restores on a weekly basis to recover deleted items for end users. If the customer moved to Exchange 2010 they could move that process into Exchange by simply increasing their mailboxes capacity for dumpster:

  • Users send/receive 100 messages per work day and have an average message size of 50KB
  • Single Item Recovery is enabled and the deleted retention window is configured to be 90 days
  • 10% of items are edited
  • Mailbox capacity calculations
    • 5 work days * 100 emails = 500 emails / week
    • For Purges:
      • 500 emails / week * 13 weeks = 6500 emails / retention period
      • 6500 emails * 50KB ? 318MB
    • For Versions:
      • 500 emails / week * 13 weeks = 6500 emails / retention period
      • 6500 emails * .1 = 650 emails
      • 650 emails * 50KB ? 32MB
    • Total Space Required per mailbox: 350MB

By increasing each mailbox’s capacity by a minimum of 350MB, backups are no longer needed for single item recovery. Single item recovery can be maintained and performed within Exchange.

But let’s not stop there. What if the requirement is that items must be recoverable for 1 year? Assuming the same assumptions used in the previous example with the exception that deleted item retention is now configured for 365 days, each mailbox needs an additional minimum 1.4GB of space.

Ultimately, if the storage subsystem is planned and designed appropriately and mailbox resiliency features are leveraged, traditional point-in-time backups can be relegated to a disaster recovery mechanism, if they are even needed at all.

Conclusion

Exchange 2010 introduces the concept of single item recovery. Single Item Recovery prevents purging of data and provides versioning capability (the ability to retain the unaltered form of the item). By default this data is retained until the age of the deleted item has exceeded the deleted item retention window. In addition, Exchange 2010 enables long-term preservation of data for litigation hold scenarios by preventing the purging of data all together.

The attempt to connect to http://yourserver/PowerShell using “Kerberos” authentication failed: Connecting to remote server failed with the following error message : WinRM cannot process the request.

The following error occurred while attempting to connect to the specified Exchange server ‘yourserver.domain.com’:The attempt to connect to http://yourserver.domain.com/PowerShell using “Kerberos” authentication failed: Connecting to remote server failed with the following error message : WinRM cannot process the request. The following error occured while using Kerberos authentication: The network path was not found.  Possible causes are:  -The user name or password specified are invalid.  -Kerberos is used when no authentication method and no user name are specified.  -Kerberos accepts domain user names, but not local user names.  -The Service Principal Name (SPN) for the remote computer name and port does not exist.  -The client and remote computers are in different domains and there is no trust between the two domains. After checking for the above issues, try the following:  -Check the Event Viewer for events related to authentication.  -Change the authentication method; add the destination computer to the WinRM TrustedHosts configuration setting or use HTTPS transport. Note that computers in the TrustedHosts list might not be authenticated.   -For more information about WinRM configuration, run the following command: winrm help config. For more information, see the about_Remote_Troubleshooting Help topic.

The solution is to first export and then delete

HKCUSoftwareMicrosoftExchangeServerv14AdminToolsNodeStructureSettings (REG_BINARY key) registry key.

What is Datacenter Activation Coordination Mode DAC in DAG in MS Exchange 2010, 2013

Datacenter Activation Coordination (DAC) mode is a property setting for a database availability group (DAG). DAC mode is disabled by default and should be enabled for all DAGs with two or more members that use continuous replication. DAC mode shouldn’t be enabled for DAGs in third-party replication mode unless specified by the third-party vendor.

If a catastrophic failure occurs that affects the DAG (for example, a complete failure of one of the datacenters), DAC mode is used to control the startup database mount behavior of a DAG. When DAC mode isn’t enabled, and a failure occurs that affects multiple servers in the DAG, when a majority of the DAG members are restored after the failure, the DAG will restart and attempt to mount databases. In a multi-datacenter configuration, this behavior could cause split brain syndrome, a condition that occurs when all networks fail, and DAG members can’t receive heartbeat signals from each other. Split brain syndrome can also occur when network connectivity is severed between the datacenters. Split brain syndrome is prevented by always requiring a majority of the DAG members (and in the case of DAGs with an even number of members, the DAG’s witness server) to be available and interacting for the DAG to be operational. When a majority of the members are communicating, the DAG is said to have quorum.

For example, consider a scenario where the first datacenter contains two DAG members and the witness server, and the second datacenter contains two other DAG members. If the first datacenter loses power and you activate the DAG in the second datacenter (for example, by activating the alternate witness server in the second datacenter), if the first datacenter is restored without network connectivity to the second datacenter, the active databases within the DAG may enter a split brain condition.

How DAC mode works

DAC mode is designed to prevent split brain from occurring by including a protocol called Datacenter Activation Coordination Protocol (DACP). After a catastrophic failure, when the DAG recovers, it won’t automatically mount databases even though the DAG has a quorum. Instead DACP is used to determine the current state of the DAG and whether Active Manager should attempt to mount the databases.

You might think of DAC mode as an application level of quorum for mounting databases. To understand the purpose of DACP and how it works, it’s important to understand the primary scenario it’s intended to deal with. Consider the two-datacenter scenario. Suppose there is a complete power failure in the primary datacenter. In this event, all of the servers and the WAN are down, so the organization makes the decision to activate the standby datacenter. In almost all such recovery scenarios, when power is restored to the primary datacenter, WAN connectivity is typically not immediately restored. This means that the DAG members in the primary datacenter will power up, but they won’t be able to communicate with the DAG members in the activated standby datacenter. The primary datacenter should always contain the majority of the DAG quorum voters, which means that when power is restored, even in the absence of WAN connectivity to the DAG members in the standby datacenter, the DAG members in the primary datacenter have a majority and therefore have quorum. This is a problem because with quorum, these servers may be able to mount their databases, which in turn would cause divergence from the actual active databases that are now mounted in the activated standby datacenter.

DACP was created to address this issue. Active Manager stores a bit in memory (either a 0 or a 1) that tells the DAG whether it’s allowed to mount local databases that are assigned as active on the server. When a DAG is running in DAC mode (which would be any DAG with three or more members), each time Active Manager starts up the bit is set to 0, meaning it isn’t allowed to mount databases. Because it’s in DAC mode, the server must try to communicate with all other members of the DAG that it knows to get another DAG member to give it an answer as to whether it can mount local databases that are assigned as active to it. The answer comes in the form of the bit setting for other Active Managers in the DAG. If another server responds that its bit is set to 1, it means servers are allowed to mount databases, so the server starting up sets its bit to 1 and mounts its databases.

But when you recover from a primary datacenter power outage where the servers are recovered but WAN connectivity has not been restored, all of the DAG members in the primary datacenter will have a DACP bit value of 0; and therefore none of the servers starting back up in the recovered primary datacenter will mount databases, because none of them can communicate with a DAG member that has a DACP bit value of 1.

Error: Object is read only because it was created by a future version of Exchange: 0.10 (14.0.100.0). Current supported version is 0.1 (8.0.535.0)

CAUSE

Previewing an object experiencing the error message, we found that we were importing the msExchVersion Attribute.  This is a Direct flow, indicating that we are pulling the information directly from the Microsoft Exchange 2010 User Object.  Confirmed in the management agent that was exporting to Microsoft Exchange 2007, that we were exporting the msExchVersion value in Direct Flow.

Configure Attribute Flow – Management Agent exporting to Exchange 2007

Generating Preview of the User Object to seethe value

RESOLUTION

We were able to resolve this issue by removing the msExchVersion attribute from the Export Attribute Flow on the management agent exporting to Microsoft Exchange 2007.

If this attribute is important to you, you may consider using a management agent extension to control the version number that goes into the msExchVersion attribute.

Service ‘MSExchangeTransport’ failed to reach status ‘Running’ on this server during installation of exchange server

This Error commom when you are installing multiple Roles on a single server at the same time.

Please try below sequence and install one role at a time.

1. Client Access Server

2. Hub Transport Server

3. Mailbox Server

This issue would not occur ever again.

Error 3221684229 when installing mailbox role of Exchange 2007 SP1 on Windows Server 2008 r2

Problem:-

Mailbox Role  Failed

Error:  An error occurred. The error code was 3221684229. The message was Access is denied..

Resolution:-

Run setup.exe in compatibilty mode. I chose Windows 2008 SP1 and then the installation went through normally.