WCM Syndication problem - Content doesn't syndicate even after creating syndication pair successfully - Network ports issue

At very first step of syndication is to check if both servers are able to communicate each other , So I just checked using "ping server1" from server2 and "ping server2" from server1 and got no pocket loss and thought both are communicating fine (Infrastructure team overlooked and confirmed that all ports are open , so didn't validated at port level initially) .

On Server1 (subscriber),
  1. Click on Subsribers --> subscribe --> provided server2 (syndicator) url information and provided the credential vault information.
  2. Then click next and able to select the libraries on the server2 (Syndicator) and choose the live items and created the syndication pair successfully.
  3. But after that actual contents are not getting syndicated and I see following error on Syndicator
[9/13/13 10:44:20:576 EDT] 00000095 ItemChangedTa E   Operation timed out for ser2_to_ser1_synd. Could be due to an invalid URL address.
  1. No error from subscriber.

Tried following on Syndicator
  1. On Server2 (Syndicator), Clicked on rebuild causes error like above
[9/13/13 10:48:35:611 EDT] 00000065 RebuildSyndic E   Operation timed out for ser2_to_ser1_synd. Could be due to an invalid URL address.
  1. No error on Subscriber.

Tried following on Subscriber
  1. On Server1(Subscriber), Clicked on rebuild causes below error
[9/13/13 10:51:05:656 EDT] 00000054 RebuildSubscr E   Could not update subscriber with name 42179ad4-49eb-47ac-aee0-54014185f390
                                 com.aptrix.syndication.IceException: IWKWC0004I: Code #400, phrase: Generic request error
        at com.aptrix.syndication.business.AbstractRequestBL.checkResponse(AbstractRequestBL.java:91)
        at com.aptrix.syndication.business.subscriber.RequestBL.updateSubscription(RequestBL.java:348)
        at com.aptrix.deployment.subscriber.PlutoSubscriber.sendUpdateRequest(PlutoSubscriber.java:313)
        at com.ibm.workplace.wcm.app.ui.admin.subscriber.RebuildSubscriberCommand.doCommand(RebuildSubscriberCommand.java:160)
        at com.ibm.workplace.wcm.app.ui.portlet.command.AbstractCommand.commandPerformed(AbstractCommand.java:209)
        at com.ibm.psw.wcl.core.CommandHandler.handleCommand(CommandHandler.java:74)
        at com.ibm.psw.wcl.core.markup.WHyperlink$EHyperlinkCallbackWrapper.handleTrigger(WHyperlink.java:482)
  ....
        at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
        at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
        at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
        at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1646)


This doesn't gave much of idea where is problem was , so we have enabled the traces to figure out the root cause

  1. Added following trace string "com.aptrix.deployment.*=all:com.aptrix.syndication.*=all" on syndicator and subscriber servers.
  2. On Server1(Subs) , clicked on rebuild subscriber causing I see the following information in trace.log

[9/12/13 13:02:38:089 EDT] 00000044 RequestBL     > RequestBL updateSubscription ENTRY
[9/12/13 13:02:38:090 EDT] 00000044 AbstractReque 3   sendRequest() - Sending request with url [http://syndserv2.sivavaka.com:10039/wps/wcm/connect/?MOD=Synd] and requestMsg com.aptrix.syndication.messaging.RequestMsg@2fb62fb6

[9/12/13 13:02:38:090 EDT] 00000044 AbstractReque 3   sendRequest() - Sending ice payload:
<ice-payload ice.version="1.0" timestamp="2013-09-12T17:02:38,90" payload-id="PL-2013-09-12T17:02:38,90-easwps07b-871323263"><ice-header><ice-user-agent>IBM Lotus Web Content Management Syndication v7.0.0.1 (bld wp7001CF11_001_23 (70x.CF28.0.1) 02/14/2012)</ice-user-agent><ice-sender sender-id="5eb10706-8368-4777-a561-0aacaf83f61f" role="subscriber" name="Serv2_To_Serv1_Subs"></ice-sender><ice-receiver receiver-id="43eb6a2e-07fb-4493-a861-8e6fe3e35d1e" name="Serv2_To_Serv1_Synd"></ice-receiver></ice-header><ice-request request-id="REQ-easwps07b-2013-09-12T17:02:38,90-871323264"><ice-get-package subscription-id="43eb6a2e-07fb-4493-a861-8e6fe3e35d1e" current-state="ICE-INITIAL"></ice-get-package></ice-request></ice-payload>

[9/12/13 13:02:38:090 EDT] 00000044 HttpMessaging 1   Generated payload to send:
<ice-payload ice.version="1.0" timestamp="2013-09-12T17:02:38,90" payload-id="PL-2013-09-12T17:02:38,90-easwps07b-871323263"><ice-header><ice-user-agent>IBM Lotus Web Content Management Syndication v7.0.0.1 (bld wp7001CF11_001_23 (70x.CF28.0.1) 02/14/2012)</ice-user-agent><ice-sender sender-id="5eb10706-8368-4777-a561-0aacaf83f61f" role="subscriber" name="Serv2_To_Serv1_Subs"></ice-sender><ice-receiver receiver-id="43eb6a2e-07fb-4493-a861-8e6fe3e35d1e" name="Serv2_To_Serv1_Synd"></ice-receiver></ice-header><ice-request request-id="REQ-easwps07b-2013-09-12T17:02:38,90-871323264"><ice-get-package subscription-id="43eb6a2e-07fb-4493-a861-8e6fe3e35d1e" current-state="ICE-INITIAL"></ice-get-package></ice-request></ice-payload>


[9/12/13 13:04:34:192 EDT] 00000044 HttpMessaging 1   Payload candidate recieved:
<ice-payload ice.version="1.0" timestamp="2013-09-12T17:02:38,145" payload-id="PL-2013-09-12T17:02:38,145-eatwps07a.nro.glic.com-1761471632"><ice-header><ice-user-agent>IBM Lotus Web Content Management Syndication v7.0.0.1 (bld wp7001CF11_001_23 (70x.CF28.0.1) 02/14/2012)</ice-user-agent><ice-sender sender-id="43eb6a2e-07fb-4493-a861-8e6fe3e35d1e" role="syndicator" name="Serv2_To_Serv1_Synd"></ice-sender><ice-receiver receiver-id="5eb10706-8368-4777-a561-0aacaf83f61f" name="Serv2_To_Serv1_Subs"></ice-receiver></ice-header><ice-response response-id="RSP-eatwps07a.nro.glic.com-2013-09-12T17:02:38,145-1761471633"><ice-code message-id="REQ-easwps07b-2013-09-12T17:02:38,90-8
71323264" phrase="Generic request error" numeric="400"></ice-code></ice-response></ice-payload>

  1. From the above, what we understood Subscriber is able to communicate with syndicator (because we got response back) .


Solution in this case
  1. We tried creating the syndication pair from the server2 (syndicator) to server1 (subscriber) just to make sure the everything fine , then we found that (server2)Syndicator is not able to communicate with subscriber(server1). (Server1 to server2 is fine but not reverse).

  1. Gone back tried to validate the open ports between the servers (not just "ping") , we tried telnet to that portal default port (10039) and found that 10039 not open . This is basic thing we should have tried this to make sure ports are open communicating over those ports are fine.

Failed communicate when we tried "telnet subsserv1.sivavaka.com 10039" from syndicator server 


NOTE: "telnet syndserv2.sivavaka.com 10039" from subscriber server is fine.

No comments:

Post a Comment