Changing Sitecore item references when creating, copying, duplicating and cloning

Reason

When copying, duplicating or cloning an item tree (an item and its descendants) references in e.g. link, rich text and tree list fields remain unchanged. This makes sense as far as references to items “outside” the duplicated item tree are concerned, but references pointing “inward” should be modified to point to the original item’s counterpart amongst the copied items.

E.g.: The item “/sitecore/content/Original” is copied to “/sitecore/content/Copy”, along with all its descendants.
Assume that “Original” contains a link to the item “/sitecore/Content/Original/Child”: even though “/sitecore/content/Copy” has an equivalent child item (located at “/sitecore/content/Copy/Child”), the link in “Copy” is still referencing “/sitecore/content/Original/Child”.

Example - Original item

Example - Copy item

The method of duplication is irrelevant; copying, cloning and duplicating all behave the same.

An example of how rich text, link, tree list, layout and similar fields can be updated to reference the proper copies is outlined in the following.
Examples are based on Sitecore 6.6 and .NET 4.5.

Code

Reacting to “duplication-events” requires hooking into the pipelines uiDuplicateItem, uiCopyItems, uiCloneItems and the item:added event. The operational flow is outlined below:

  1. A pipeline processor creates, copies, duplicates or clones an item, or the item:added event is triggered.
  2. The source and target of the operation is passed to a ReferenceReplacementJob which uses the Sitecore.Jobs.JobManager to run asynchronously.
  3. An ItemPathTranslator is used to translate between source and target item paths.
  4. Pairs of source and target items, found using the translated paths, are added to an ItemReferenceReplacer.
  5. Once all pairs have been added, the reference replacer processes all “non-Sitecore fields” and the layout field (“__Renderings”). All ID’s, paths etc. from the “source item tree” are replaced with their equivalents from the “target item tree”.

To overwrite and/or amend the Web.config file with the required pipeline processors, save the following to a .config-file and place it in the “App_Config/Include”-folder. Replace namespace and assembly names as needed.

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <events>
      <event name="item:added">
        <handler type="NamespaceName.AddFromTemplate, AssemblyName" method="OnItemAdded" />
      </event>
    </events>

    <processors>
      <uiDuplicateItem>
        <processor mode="on" type="Sitecore.Shell.Framework.Pipelines.DuplicateItem,Sitecore.Kernel" method="Execute">
          <patch:attribute name="type">NamespaceName.DuplicateItem, AssemblyName</patch:attribute>
          <patch:attribute name="method">Execute</patch:attribute>
        </processor>
      </uiDuplicateItem>

      <uiCopyItems>
        <processor mode="on" type="NamespaceName.CopyOrCloneItem, AssemblyName" method="ProcessFieldValues" />
      </uiCopyItems>

      <uiCloneItems>
        <processor mode="on" type="NamespaceName.CopyOrCloneItem, AssemblyName" method="ProcessFieldValues" />
      </uiCloneItems>
    </processors>
  </sitecore>
</configuration>

The standard DuplicateItem processor has to be replaced with a slightly modified version, due to the lack of information it conveys to later pipeline steps: it’s not possible to determine the source and target of a “duplicate item”-operation. The standard implementation of the copy and clone-processors do not have this problem, which simplifies amendment of the corresponding pipelines.

The DuplicateItem-processor is based on the inner workings of Sitecore.Shell.Framework.Pipelines.DuplicateItem, with just enough modifications to create a ReferenceReplacementJob of off the source and target.

using Sitecore;
using Sitecore.Configuration;
using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.Diagnostics;
using Sitecore.Globalization;
using Sitecore.Web.UI.Sheer;

public class DuplicateItem
{
  private Item _itemToDuplicate;

  public new void Execute(ClientPipelineArgs args)
  {
    Item copy = Duplicate(args);
    if (copy == null)
      return;

    if (_itemToDuplicate == null)
      return;

    new ReferenceReplacementJob(_itemToDuplicate, copy).StartAsync();
  }

  private Item Duplicate(ClientPipelineArgs args)
  {
    Assert.ArgumentNotNull(args, "args");
    Item item = GetItemToDuplicate(args);
    _itemToDuplicate = item;
    if (item == null)
    {
      SheerResponse.Alert("Item not found.", new string[0]);
      args.AbortPipeline();
    }
    else
    {
      Item parent = item.Parent;
      if (parent == null)
      {
        SheerResponse.Alert("Cannot duplicate the root item.", new string[0]);
        args.AbortPipeline();
      }
      else if (parent.Access.CanCreate())
      {
        Log.Audit(this, "Duplicate item: {0}", new string[] { AuditFormatter.FormatItem(item) });
        return Context.Workflow.DuplicateItem(item, args.Parameters["name"]);
      }
      else
      {
        SheerResponse.Alert(Translate.Text("You do not have permission to duplicate \"{0}\".", new object[] { item.DisplayName }), new string[0]);
        args.AbortPipeline();
      }
    }
    return null;
  }

  private Item GetItemToDuplicate(ClientPipelineArgs args)
  {
    Language language;
    Database database = Factory.GetDatabase(args.Parameters["database"]);
    Assert.IsNotNull(database, args.Parameters["database"]);
    string str = args.Parameters["id"];
    if (!Language.TryParse(args.Parameters["language"], out language))
    {
      language = Context.Language;
    }
    return database.GetItem(ID.Parse(str), language);
  }
}

Both the copy and clone pipelines are provided with CopyItemsArgs containing source and target of the “duplication-event”, allowing for a clean extension of the respective pipelines (rather than forcing decompilation/rewriting of existing processors).

using System.Linq;
using Sitecore.Data.Items;
using Sitecore.Diagnostics;
using Sitecore.Shell.Framework.Pipelines;

public class CopyOrCloneItem : Sitecore.Shell.Framework.Pipelines.CopyItems
{
  public virtual void ProcessFieldValues(CopyItemsArgs args)
  {
    Item sourceRoot = base.GetItems(args).FirstOrDefault();
    Assert.IsNotNull(sourceRoot, "sourceRoot is null.");

    Item copyRoot = args.Copies.FirstOrDefault();
    Assert.IsNotNull(copyRoot, "copyRoot is null.");

    new ReferenceReplacementJob(sourceRoot, copyRoot).StartAsync();
  }
}

The following handler is courtesy of Wilhelm who suggested using the item:added event instead of the uiAddFromTemplate pipeline, which allows for clean extension and handling of branch-based item creation.

using Sitecore.Data.Items;
using Sitecore.Events;
using System;

public class AddFromTemplate
{
  public void OnItemAdded(object sender, EventArgs args)
  {
    Item targetItem = Event.ExtractParameter(args, 0) as Item;
    if (targetItem == null)
      return;
    if (targetItem.Branch == null)
      return;
    if (targetItem.Branch.InnerItem.Children.Count != 1)
      return;
    Item branchRoot = targetItem.Branch.InnerItem.Children[0];
    new ReferenceReplacementJob(branchRoot, targetItem).StartAsync();
  }
}

The ReferenceReplacementJob retrieves all source and target items using an ItemPathTranslator, and eventually lets an ItemReferenceReplacer process all target items.

using Sitecore;
using Sitecore.Data.Fields;
using Sitecore.Data.Items;
using Sitecore.Data.Query;
using Sitecore.Diagnostics;
using Sitecore.Jobs;
using System.Collections.Generic;
using System.Linq;

public class ReferenceReplacementJob
{
  private readonly Item _source;
  private readonly Item _target;

  public ReferenceReplacementJob(Item source, Item target)
  {
    Assert.ArgumentNotNull(source, "source");
    Assert.ArgumentNotNull(target, "target");
    _source = source;
    _target = target;
  }

  public void StartAsync()
  {
    string jobCategory = typeof(ReferenceReplacementJob).Name;
    string siteName = Context.Site == null ? "No Site Context" : Context.Site.Name;
    JobOptions jobOptions = new JobOptions(GetJobName(), jobCategory, siteName, this, "Start");
    JobManager.Start(jobOptions);
  }

  private string GetJobName()
  {
    return string.Format("Resolving item references between source {0} and target {1}.", AuditFormatter.FormatItem(_source), AuditFormatter.FormatItem(_target));
  }

  public void Start()
  {
    ItemPathTranslator translator = new ItemPathTranslator(_source, _target);
    IEnumerable<Item> sourceDescendants = GetDescendantsAndSelf(_source);
    ItemReferenceReplacer replacer = InitializeReplacer(sourceDescendants, translator);
    foreach (Item equivalentTarget in replacer.OtherItems)
    {
      replacer.ReplaceItemReferences(equivalentTarget);
    }
  }

  private IEnumerable<Item> GetDescendantsAndSelf(Item source)
  {
    return Query.SelectItems("descendant-or-self::*", source) ?? Enumerable.Empty<Item>();
  }

  private ItemReferenceReplacer InitializeReplacer(IEnumerable<Item> sourceDescendants, ItemPathTranslator translator)
  {
    ItemReferenceReplacer replacer = new ItemReferenceReplacer(ExcludeStandardSitecoreFieldsExceptLayout);
    foreach (Item sourceDescendant in sourceDescendants)
    {
      if (!translator.CanTranslatePath(sourceDescendant))
        continue;

      Item equivalentTarget = sourceDescendant.Database.GetItem(translator.TranslatePath(sourceDescendant));
      if (equivalentTarget == null)
        continue;

      replacer.AddItemPair(sourceDescendant, equivalentTarget);
    }
    return replacer;
  }

  private bool ExcludeStandardSitecoreFieldsExceptLayout(Field field)
  {
    Assert.ArgumentNotNull(field, "field");
    return field.ID == FieldIDs.LayoutField || !field.Name.StartsWith("__");
  }
}

Item paths are “translated” relative to a common root item (e.g. a site root item in a multi-site solution, or the content root in a single-site solution).

using Sitecore;
using Sitecore.Data.Items;
using Sitecore.Diagnostics;
using Sitecore.Exceptions;

public class ItemPathTranslator
{
  private readonly Item _source;
  private readonly Item _sourceRoot;
  private readonly Item _target;
  private readonly Item _targetRoot;

  public ItemPathTranslator(Item source, Item target)
    : this(source, source, target, target)
  {
  }

  public ItemPathTranslator(Item source, Item sourceRoot, Item target, Item targetRoot)
  {
    Assert.ArgumentNotNull(source, "source");
    Assert.ArgumentNotNull(sourceRoot, "sourceRoot");
    Assert.ArgumentNotNull(target, "target");
    Assert.ArgumentNotNull(targetRoot, "targetRoot");
    _source = source;
    _sourceRoot = sourceRoot;
    _target = target;
    _targetRoot = targetRoot;
  }

  public bool CanTranslatePath(Item item)
  {
    Assert.ArgumentNotNull(item, "item");
    return IsDescendantOrSelf(item, _source) || IsDescendantOrSelf(item, _sourceRoot);
  }

  private bool IsDescendantOrSelf(Item item, Item otherItem)
  {
    return item.ID == otherItem.ID || item.Axes.IsDescendantOf(otherItem);
  }

  public string TranslatePath(Item item)
  {
    Assert.ArgumentNotNull(item, "item");
    if (IsDescendantOrSelf(item, _source))
      return GetFullPath(_target, _source, item);

    if (IsDescendantOrSelf(item, _sourceRoot))
      return GetFullPath(_targetRoot, _sourceRoot, item);

    throw new InvalidItemException(string.Format("Item {0} is not a descendant of {1} or {2}.",
      AuditFormatter.FormatItem(item),
      AuditFormatter.FormatItem(_source),
      AuditFormatter.FormatItem(_sourceRoot)));
  }

  private string GetFullPath(Item closestEquivalentAncestor, Item closestAncestor, Item item)
  {
    string startPathPart = closestEquivalentAncestor.Paths.FullPath;
    string relativePathPart = GetRelativePath(closestAncestor, item);
    return StringUtil.EnsurePostfix('/', startPathPart) + StringUtil.RemovePrefix('/', relativePathPart);
  }

  private string GetRelativePath(Item closestAncestor, Item item)
  {
    return item.Paths.FullPath.Replace(closestAncestor.Paths.FullPath, string.Empty);
  }
}

Item references are stored in different formats in Sitecores various field types, hence reference replacement is performed in four ways: ID, ShortID, full path and content path.
Using this “broad spectrum” approach allows proper replacement of references in all relevant  field types (general link, rich text, layout, tree list etc.).

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Text;
using Sitecore.Data.Fields;
using Sitecore.Data.Items;
using Sitecore.Diagnostics;

public class ItemReferenceReplacer
{
  private readonly Func<Field, bool> _fieldFilter;
  private readonly ICollection<ItemPair> _itemPairs = new HashSet<ItemPair>();

  public IEnumerable<Item> Items
  {
    get { return _itemPairs.Select(pair => pair.Item).ToArray(); }
  }

  public IEnumerable<Item> OtherItems
  {
    get { return _itemPairs.Select(pair => pair.OtherItem).ToArray(); }
  }

  public ItemReferenceReplacer(Func<Field, bool> fieldFilter)
  {
    Assert.ArgumentNotNull(fieldFilter, "fieldFilter");
    _fieldFilter = fieldFilter;
  }

  public void AddItemPair(Item item, Item otherItem)
  {
    Assert.ArgumentNotNull(item, "item");
    Assert.ArgumentNotNull(otherItem, "otherItem");
    _itemPairs.Add(new ItemPair(item, otherItem));
  }

  public void ReplaceItemReferences(Item item)
  {
    IEnumerable<Field> fields = GetFieldsToProcess(item);
    foreach (Field field in fields)
    {
      foreach (Item itemVersion in GetVersionsToProcess(item))
      {
        Field itemVersionField = itemVersion.Fields[field.ID];
        ProcessField(itemVersionField);
      }
    }
  }

  private IEnumerable<Field> GetFieldsToProcess(Item item)
  {
    item.Fields.ReadAll();
    return item.Fields.Where(_fieldFilter).ToArray();
  }

  private IEnumerable<Item> GetVersionsToProcess(Item item)
  {
    return item.Versions.GetVersions(true);
  }

  private void ProcessField(Field field)
  {
    string initialValue = GetInitialFieldValue(field);
    if (string.IsNullOrEmpty(initialValue))
      return;

    StringBuilder value = new StringBuilder(initialValue);
    foreach (ItemPair itemPair in _itemPairs)
    {
      ReplaceID(itemPair.Item, itemPair.OtherItem, value);
      ReplaceShortID(itemPair.Item, itemPair.OtherItem, value);
      ReplaceFullPath(itemPair.Item, itemPair.OtherItem, value);
      ReplaceContentPath(itemPair.Item, itemPair.OtherItem, value);
    }
    UpdateFieldValue(field, initialValue, value);
  }

  private string GetInitialFieldValue(Field field)
  {
    return field.GetValue(true, true);
  }

  private void ReplaceID(Item item, Item otherItem, StringBuilder value)
  {
    value.Replace(item.ID.ToString(), otherItem.ID.ToString());
  }

  private void ReplaceShortID(Item item, Item otherItem, StringBuilder value)
  {
    value.Replace(item.ID.ToShortID().ToString(), otherItem.ID.ToShortID().ToString());
  }

  private void ReplaceFullPath(Item item, Item otherItem, StringBuilder value)
  {
    value.Replace(item.Paths.FullPath, otherItem.Paths.FullPath);
  }

  private void ReplaceContentPath(Item item, Item otherItem, StringBuilder value)
  {
    if (item.Paths.IsContentItem)
      value.Replace(item.Paths.ContentPath, otherItem.Paths.ContentPath);
  }

  private void UpdateFieldValue(Field field, string initialValue, StringBuilder value)
  {
    if (initialValue.Equals(value.ToString()))
      return;

    using (new EditContext(field.Item))
    {
      field.Value = value.ToString();
    }
  }

  [DebuggerDisplay("Item: \"{Item.Paths.Path}\", OtherItem: \"{OtherItem.Paths.Path}\"")]
  private class ItemPair
  {
    public Item Item { get; private set; }
    public Item OtherItem { get; private set; }

    public ItemPair(Item item, Item otherItem)
    {
      Assert.ArgumentNotNull(item, "item");
      Assert.ArgumentNotNull(otherItem, "otherItem");
      Item = item;
      OtherItem = otherItem;
    }

    public override bool Equals(object instance)
    {
      return instance is ItemPair && instance.GetHashCode() == GetHashCode();
    }

    public override int GetHashCode()
    {
      return string.Concat(Item.ID, OtherItem.ID).GetHashCode();
    }
  }
}

Example

Example of droptree and treelist field. The field contents have been properly updated by ReplaceID.

Example - droptree and treelist field in original

Example - droptree and treelist field in copy

Example of rich text and general link field. The field contents have been properly updated by ReplaceID, ReplaceShortID and ReplaceContentPath.

Example - rich text and general link in original

Example - rich text and general link in copy

Example of data source path in layout field. The field content has been properly updated by ReplaceFullPath. Data source IDs would also be replaced correctly, hence the replacement process is compatible with Convert Data Source Paths to IDs and similar solutions.

Example - layout field in original

Example - layout field in copy

19 thoughts on “Changing Sitecore item references when creating, copying, duplicating and cloning

    • All code required to make it work should be available from the snippets in this post. If a piece of the puzzle is missing, you’ll have to point it out to me.

  1. Very nice article. I needed some of this function when creating a item from a branch. I dont think its a good idea to patch Sitecore.Shell.Framework.Pipelines.AddFromTemplate in case sitecore is doing changes to this method. I think a better way to do this is using the event item:added
    and the folowing code.

    • Hi Wilhelm,
      I didn’t consider moving the logic to the non-UI pipeline, it’s a much cleaner solution – thanks for the input, I’ll update the code accordingly!

  2. protected void OnItemAdded(object sender, EventArgs args)
    {
    Sitecore.Data.Items.Item targetItem = Sitecore.Events.Event.ExtractParameter(args, 0) as Sitecore.Data.Items.Item;
    if (targetItem == null)
    return;
    if (targetItem.Branch != null)
    new ReferenceReplacementJob(targetItem, targetItem.Branch).StartAsync();
    }
    Atleast this works in Sitecore 7.0 not tested in any earlier versions yet.

  3. Sorry about my last post it was wrong. The right code is
    Sitecore.Data.Items.Item targetItem = Sitecore.Events.Event.ExtractParameter(args, 0) as Sitecore.Data.Items.Item;
    if (targetItem == null)
    return;
    if (targetItem.Branch == null)
    return;
    if (targetItem.Branch.InnerItem.Children.Count == 1)
    {
    Sitecore.Data.Items.Item branchRoot = targetItem.Branch.InnerItem.Children[0];
    new DatasourceReferenceReplacementJob(branchRoot,targetItem).StartAsync();
    }
    This code only works with one item under the branch

  4. Thanks for the post. I’ve used it on a recent project where we had to change the datasource in the presentation for each of the page components that were relative to the page and had to be updated when a user was copying and duplicating items.

  5. Great post. Thank you! Only one note. Sitecore 7 injects another processor (Sitecore.Buckets.Pipelines.UI.ItemDuplicate) right before the legacy Sitecore.Shell.Framework.Pipelines.DuplicateItem. It does a little more than just calling the Context.Workflow.DuplicateItem to support buckets. Unfortunately it doesn’t fix the issue with the arguments and still doesn’t carry the copy down the pipeline. It actually aborts the pipeline (so there’s no pipeline after the new ItemDuplicate executes) which is probably just not the best design choice (path:before + abort pipeline is not as clean as patch:instead)

  6. To my last comment. Actually all buckets-aware processors that are injected via Sitecore.Buckets.config abort the pipeline. The Duplicate is the one that truly runs instead of the legacy version for all items wether they are bucketed or not. The Copy/Clone lets the rest of the pipeline run but will take over and then abort for the bucketed items. Need to look at 7.1 but it is this way in 7.0

    • Thank you for the heads up! The particular client for whom much of this code was originally written hasn’t upgraded to Sitecore 7 yet, so I’ve been oblivious to this issue so far.

  7. Pingback: A Tale of three friends. Close look at three Sitecore pipelines. | Jocks to the Core

  8. Hi Uli,

    I really appreciate your work and effort you take to share solution with us. I have implemented in our environment with little changes as below.

    Scenario client wants same website like parent, including sharing of content but with conditional rendering of domain. Thus we choose to select clone option.
    1. Replace below line
    //return Query.SelectItems(“descendant-or-self::*”, source) ?? Enumerable.Empty();
    //This will only return 100 items.
    with

    List selfOrDecendents = new List();
    selfOrDecendents.Add(source);
    selfOrDecendents.AddRange(source.Axes.GetDescendants()); //Expensive but works
    return selfOrDecendents;

    • Hi Mrunal,

      It sounds as if Query.MaxItems is set to the default value of 100 in your solution (search for “Query.MaxItems” in Web.config).
      It’s a setting which limits the number of items returned from any Sitecore query. Try changing it to e.g. 1000 and see what happens when you run the original code. Setting it to 0 will remove the limit and always return all results from a query.

      Good luck and happy coding!

  9. Thanks Uli,

    I believe Query.MaxItems =100 is for better performance and I don’t want to touch that stuffs because once you change it, it will return specified amount of data every time for every request and also sitecore environment, which we might not required.

    In our case we cloned website once in a while, thus I will stick to API, to avoid performance penalty

    Thanks again :)..

    • Hi Roland,

      An item tree 4 levels deep will be handled like any other item tree, there are no restrictions regarding the level of nesting. If you find that some of the code doesn’t work for items below a certain depth, I suggest checking your web.config for e.g. the MaxTreeDepth-setting and ensuring that it’s set according to your expectations.

      Items created from branches are handled by the AddFromTemplate pipeline processor. Search for “branch” and you should find it eventually.

      • One note on the branch templates. The ItemReferenceReplacer won’t create ItemPair for the child nodes with $name token in them. The reason is – ItemPathTranslator won’t replace the token and won’t find the item. As a result those child nodes will be skipped and will keep the links pointing to the original structure. A very easy fix is to add relativePathPart.Replace($name, _target.Name) in the ItemPathTranslator.GetFullPath(). Hope it helps!

      • Hi Pavel,

        Thanks for the info! I misunderstood the initial question, not realizing that it was concerning copying a branch itself, rather than using it to create new content.
        I’ve only ever used the code posted here for copying “actual content”, not branch items, template items and the like. It’s a bit surprising that the expandInitialFieldValue and expandBranchItemName pipelines aren’t triggered when copying or creating items using this code.

  10. Hi Uli,

    Thanks for this example. It is still working on Sitecore 8.1. The only thing I had to adapt was to include the FinalLayout field in the ExcludeStandSitecoreFieldsExceptLayout method.

    private bool ExcludeStandardSitecoreFieldsExceptLayout(Field field)
    {
    Assert.ArgumentNotNull(field, “field”);
    return field.ID == FieldIDs.LayoutField || field.ID == FieldIDs.FinalLayoutField || !field.Name.StartsWith(“__”);
    }

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s