Reason

The date time indexing logic that ships with Sitecore 7 Content Search is partially flawed. Neither the logic storing nor retrieving DateTime values is capable of handling changes to ContentSearch.DateFormat. This means that DateTime values will lose any information reg. hours, minutes, seconds etc. when being stored in the indices, no matter what date time format is specified in the content search configuration.

Shown below is a default and a patched ContentSearch.DateFormat. The format has been changed from “yyyyMMdd” to “s” (top vs. bottom screenshot):

The standard DateFieldReader always stores the field contents with a date resolution of DAY, hence ignoring the above setting. This means that out of the box, the hour, minute and second parts of a Sitecore DateField will not be stored regardless of the ContentSearch.DateFormat used.

Fortunately it’s easy to replace the default DateFieldReader with a custom implementation via the content search configuration files.
Unfortunately, once a DateFieldReader has been implemented which respects the ContentSearch.DateFormat, format exceptions will be thrown when working with types derived from Sitecore.ContentSearch.SearchTypes.SearchResultItem (“String was not recognized as a valid DateTime”).
This occurs when the properties CreatedDate and Updated are read from the index. These two properties are backed by index fields which are configured with a custom date time format as follows:

It turns out that the default date time converter doesn’t handle date time formats that are configured on individual fields, and there’s no obvious way to retrieve the custom storage format once a field’s value has been passed to a TypeConverter for conversion:

The following article describes a fix to the issues outlined above by overriding the default DateFieldReader and IndexFieldDateTimeValueConverter, in addition to introducing a setting for “alternate date time formats”.

Examples are based on .NET 4.5 and Sitecore 7.1 rev. 130926.

Code

The storage and retrieval process can be roughly outlined as follows:

1. A crawler reads items from Sitecore.
2. Each item’s fields are read by matching FieldReader implementations (e.g. DateFieldReader, CheckboxFieldReader, ImageFieldReader).
3. The field value is converted to a string and stored in the index.
4. When retrieving content directly from the index (i.e. when using a class containing properties marked with the Sitecore.ContentSearch.IndexFieldAttribute and Linq-to-Sitecore) a TypeConverter parses the stored value to an instance of the appropriate runtime type.

Configuration

To replace the default FieldReader and TypeConverter with our custom implementations, save the configuration shown below to a .config-file (e.g. “z.ContentSearch.DateTimeFix.config”) and place it in the “App_Config/Include”-folder.
It’s important to note that the configuration file has been prefixed with the letter “z”. This is done to ensure that it’s loaded after the configuration file “Sitecore.ContentSearch.Lucene.DefaultIndexConfiguration.config”, which is also located in the “App_Config/Include”-folder. Our configuration won’t have any effect otherwise, as Sitecore loads config include files in lexicographical order (our configuration would be overridden by the default config).
Modify namespace and assembly names as needed.

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <settings>
      <!-- The default date time format used when storing values in the index. -->
      <setting name="ContentSearch.DateFormat">
        <patch:attribute name="value">s</patch:attribute>
      </setting>
      <!-- Alternate date time formats that are used in the index. Multiple values must be separated by pipes. -->
      <setting name="ContentSearch.AlternateDateTimeFormats" value="yyyyMMdd" />
    </settings>

    <contentSearch>
      <configuration>
        <defaultIndexConfiguration>
          <!-- DateTimeFieldReader -->
          <fieldReaders type="Sitecore.ContentSearch.FieldReaders.FieldReaderMap, Sitecore.ContentSearch">
            <mapFieldByTypeName hint="raw:AddFieldReaderByFieldTypeName">
              <fieldReader fieldTypeName="date|datetime">
                <patch:attribute name="fieldReaderType">NamespaceName.DateTimeFieldReader, AssemblyName</patch:attribute>
              </fieldReader>
            </mapFieldByTypeName>
          </fieldReaders>

          <!-- DateTimeConverter -->
          <indexFieldStorageValueFormatter type="Sitecore.ContentSearch.LuceneProvider.Converters.LuceneIndexFieldStorageValueFormatter, Sitecore.ContentSearch.LuceneProvider">
            <converters hint="raw:AddConverter">
              <converter handlesType="System.DateTime">
                <patch:attribute name="typeConverter">NamespaceName.DateTimeConverter, AssemblyName</patch:attribute>
              </converter>
            </converters>
          </indexFieldStorageValueFormatter>
        </defaultIndexConfiguration>
      </configuration>
    </contentSearch>
  </sitecore>
</configuration>

FieldReader

The DateTimeFieldReader simply makes use of the ContentSearch.IndexDateFormat when invoking DateTime.ToString(...):

using System;
using Sitecore.ContentSearch;
using Sitecore.ContentSearch.FieldReaders;
using Sitecore.ContentSearch.Utilities;
using Sitecore.Data.Fields;

public class DateTimeFieldReader : FieldReader
{
  public override object GetFieldValue(IIndexableDataField field)
  {
    if (field.Value is DateTime)
      return Format((DateTime)field.Value);

    Field dataField = field as SitecoreItemDataField;
    if (dataField == null)
      return string.Empty;

    if (string.IsNullOrEmpty(dataField.Value))
      return string.Empty;

    if (FieldTypeManager.GetField(dataField) is DateField)
      return Format(new DateField(dataField).DateTime);

    return string.Empty;
  }

  private string Format(DateTime dateTime)
  {
    return dateTime.ToString(ContentSearchConfigurationSettings.IndexDateFormat);
  }
}

TypeConverter

Note that the DateTimeConverter makes use of the new ContentSearch.AlternateDateTimeFormats setting. Being forced to maintain all index date time formats in two separate parts of the configuration (as part of the field configurations themselves and as a setting) is suboptimal, but it’s a suitably pragmatic solution which will work until Sitecore provides a proper fix:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Globalization;
using Sitecore.Configuration;
using Sitecore.ContentSearch.Utilities;

public class DateTimeConverter : System.ComponentModel.DateTimeConverter
{
  private readonly List<string> _alternateDateTimeFormats = new List<string>();
  private readonly string _defaultDateTimeFormat = ContentSearchConfigurationSettings.IndexDateFormat;

  public DateTimeConverter()
  {
    _alternateDateTimeFormats.Add(_defaultDateTimeFormat);
    _alternateDateTimeFormats.AddRange(Settings.GetSetting("ContentSearch.AlternateDateTimeFormats", string.Empty).Split('|'));
  }

  public override object ConvertFrom(ITypeDescriptorContext context, CultureInfo culture, object value)
  {
    string fieldValue = value as string;
    if (string.IsNullOrEmpty(fieldValue))
      return DateTime.MinValue;

    try
    {
      return DateTime.ParseExact(fieldValue, _alternateDateTimeFormats.ToArray(), culture.DateTimeFormat, DateTimeStyles.None);
    }
    catch (FormatException ex)
    {
      string message = string.Format("{0} Value is '{1}'.", ex.Message, value);
      throw new ArgumentException(message, "value", ex);
    }
  }

  public override object ConvertTo(ITypeDescriptorContext context, CultureInfo culture, object value, Type destinationType)
  {
    return ((DateTime) value).ToString(_defaultDateTimeFormat, culture.DateTimeFormat);
  }
}

Example

Shown below is a simple C# class inheriting from the aforementioned SearchResultItem:

The following makes up the code behind of a simple ASP.NET test page (SearchResults is a GridView control):

As can be seen in the output, both the default fields __smallCreatedDate and __smallUpdatedDate as well as the field NewsDate are retrieved properly:

The screenshot below is taken from Luke, showing how the field values are stored in Lucene: