Taiyeb Zakir
Microsoft SQL Server Escalation Support Services
I recently worked on a case where Merge agent was failing with this error:
>>>
2018-04-09 19:44:40.123 The merge process is retrying a failed operation made to article 'Project' - Reason: 'Invalid date format'.
2018-04-09 19:44:40.123 OLE DB Distributor 'USSECCMPSQCE202INST2': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2018-04-09 19:44:40.311 OLE DB Subscriber 'DERUSCMPSQCE103INST2': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2018-04-09 19:44:40.389 Percent Complete: 0
2018-04-09 19:44:40.389 The merge process could not enumerate changes at the 'Publisher'. When troubleshooting, restart the synchronization with verbose history logging and specify an output file to which to write.The merge process is retrying a failed operation made to article 'Project' - Reason: 'Invalid date format'.
2018-04-09 19:44:40.389
>>>
We needed to find out what row was generating the "Invalid date format" error.
We increased reporting level for replmerg.log and saw this in the replmerg.log on the Subscriber
>>>
Oledbcon , 2018/05/09 14:37:56.284, 16988, 4357, S5, PROFILER:0 Spid:221 , Srv:USSECCMPSQCE202INST2, Db:distribution , RPC:{call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
Oledbcon , 2018/05/09 14:37:56.315, 5648, 4357, S5, PROFILER:0 Spid:271 , Srv:USSECCMPSQCE202INST2, Db:SQTool , RPC:create table #retry_table_agent (tablenick int NOT NULL, rowguid uniqueidentifier ROWGUIDCOL default newid() not null, generation int NOT NULL, errcode int NOT NULL, errtext nvarchar(255) NULL, type tinyint NOT NULL)
Oledbcon , 2018/05/09 14:37:56.424, 5648, 4357, S5, PROFILER:0 Spid:271 , Srv:USSECCMPSQCE202INST2, Db:SQTool , RPC:create procedure dbo.#insert_retry_proc_agent (
@tablenick int,
@rowguid uniqueidentifier,
@generation int,
@errcode int,
@errtext nvarchar(255),
@type tinyint ) as
update #retry_table_agent set tablenick=@tableni
Oledbcon , 2018/05/09 14:37:56.518, 5648, 7257, S5, PROFILER:1 Spid:271 , Param1 , DBTYPE_I4 , Len:4 , Val:96089000
Oledbcon , 2018/05/09 14:37:56.518, 5648, 7257, S5, PROFILER:1 Spid:271 , Param2 , DBTYPE_GUID , Len:16 , Val:cb258cdd-933f-e811-80f6-549f35cdbf10
Oledbcon , 2018/05/09 14:37:56.518, 5648, 7257, S5, PROFILER:1 Spid:271 , Param3 , DBTYPE_I8 , Len:8 , Val:3039010000000000
Oledbcon , 2018/05/09 14:37:56.518, 5648, 7257, S5, PROFILER:1 Spid:271 , Param4 , DBTYPE_I4 , Len:4 , Val:0
Oledbcon , 2018/05/09 14:37:56.518, 5648, 7257, S5, PROFILER:1 Spid:271 , Param5 , DBTYPE_WVARCHAR , Len:38 , Val:Invalid date format
Oledbcon , 2018/05/09 14:37:56.518, 5648, 7257, S5, PROFILER:1 Spid:271 , Param6 , DBTYPE_I1 , Len:1 , Val:02
Oledbcon , 2018/05/09 14:37:56.518, 5648, 4357, S5, PROFILER:0 Spid:271 , Srv:USSECCMPSQCE202INST2, Db:SQTool , RPC:{call [#insert_retry_proc_agent] (?,?,?,?,?,? )}
Oledbcon , 2018/05/09 14:37:56.628, 5648, 7257, S5, PROFILER:1 Spid:197 , Param2 , DBTYPE_I4 , Len:4 , Val:49
Oledbcon , 2018/05/09 14:37:56.628, 5648, 7257, S5, PROFILER:1 Spid:197 , Param3 , DBTYPE_I4 , Len:4 , Val:3
Oledbcon , 2018/05/09 14:37:56.628, 5648, 7257, S5, PROFILER:1 Spid:197 , Param4 , DBTYPE_WVARCHAR , Len:214 , Val:The merge process is retrying a failed operation made to article 'Project' - Reason: 'Invalid date format'.
>>>
Error was coming from Project table with rowguid 'cb258cdd-933f-e811-80f6-549f35cdbf10'.
We ran SELECT query for that rowguid checking row on the Publisher and Subscriber, but row looked okay.
Looking at table scheme, found Project table had nvarchar(max) column, considered BLOB data. We found that Merge agent performs blob optimization by default for BLOB data and here it modified the metadata incorrectly which caused this error. For more details on blob optimization check this doc and look for @stream_blob_columns.
To resolve the issue we can use either of these workarounds:
- Change nvarchar(max) to nvarchar(fixed length)
- Change @stream_blob_columns to false
- Don't publish nvarchar(max) column
We changed @stream_blob_columns to false and that resolved the issue.
DECLARE @publication AS sysname;
DECLARE @article AS sysname;
SET @publication = N'test_pub';
SET @article = N'ReplTest';
USE SQLTool
EXEC sp_changemergearticle
@publication = @publication,
@article = @article,
@property = N'stream_blob_columns',
@value = N'false';
GO