SBL-NET-01034: The SISNAPI connection was closed by the peer





Applies to:


Siebel Finance Sales - Version: 8.1 SIA [21039] and later   [Release: V8 and later ]
Information in this document applies to any platform.



Symptoms


The customer upgraded their application from 8.0.0.5 to 8.0.0.9 and experienced the following issues:

a) The OM (Object Manager) stopped responding from time to time.
b)
All the logged in users were kicked out and no user could login
thereafter. The server busy error message was encountered and the
workaround was to restart the OM.



There were no exact sequence of steps to reproduce this issue. The error
was intermittent and was happening occasionally and whenever it
happened, it kicked out users and then no user could login.

The OM logs showed the following error:

SBL-NET-01034: The SISNAPI connection was closed by the peer


Cause


The customer was using Message Broadcast feature in their environment
and the issue was caused because of the incorrect parameter setting. The
problem was caused by the parameter "Application Enable Message
Broadcast Cache" setting which was reset to False. The customer was
using this setting to maintain keep alive.  Once the parameter was reset
to true, the issue was fixed.


The cause of the issue is justified as the bookshelf also mentions
something related to the parameter, however it does not explicitly
mentions that the OM will intermittently stop responding and the users
will be kicked off.



There are two methods for ways for the application to obtain the messages for display:



a) The default behavior is to have the messages read from the database
each time the message bar is refreshed. This method can adversely affect
performance if the message bar is set to refresh frequently.

b) Message Broadcast Caching stores messages in each application’s
object manager. The messages are then broadcast through the Service
Request Broker (SRBroker).



If this is not set to TRUE, it results in reads to the database.  That explains the traffic and the subsequent issues.


Solution


The following is the solution action plan for this issue:



1) Navigate to the Administration - Server Configuration screen > Enterprises view.

2) From the Enterprise Servers list, select the enterprise server that
runs the application whose message broadcasting settings you want to
modify.

3) Click the Component Definitions tab, then select a component whose Component Type value is Application Object Manager.

4) Select the Application Enable Message Broadcast Cache parameter in
the Component Parameters subview, and type the value as TRUE.




















Applies to:


Siebel System Software - Version: 7.5.3.9 [16194] and later   [Release: V7 and later ]
Oracle Solaris on SPARC (64-bit)

Product Release: V7 (Enterprise)

Version: 7.5.3.9 [16194]

Database: Oracle 9.2.0.2

Application Server OS: Sun Solaris 8

Database Server OS: Sun Solaris 8



This document was previously published as Siebel SR 38-2884485569.



Symptoms


SBL-SVR-00027we lost accesses to the Server Admin screen and srvrmgr on multiple environment (dev, test,
staging, etc).

when accessing the server admin screen, we got error msg:
SBL-NET-01034:
The SISNAPI connection was closed by the peer

when accessing srvrmgr, we got
msg:
Connected to 0 server(s) out of a total of 1 server(s) in the enterprise

Failed to
connect server s3_intf_s1: Handshake failed

The srvrmgr.llog file shows:
2021
2006-02-07 10:11:35 2006-02-07 10:12:58 -0600 00000004 001 001f 0001 09
srvrmgr 29510 1
/u00/siebel/siebsrvr/log/srvrmgr.log 7.5.3.9 [16194]
ENU
GenericLog      GenericError    1       2006-02-07
10:11:35     (sasess.cpp 9(66
4) err=1801034 sys=0) SBL-NET-01034:
The SISNAPI connection was closed by the
pe
er
GenericLog      GenericError    1       2006-02-07
10:11:35     (sasess.cpp 9(66
4) err=902047 sys=0) SBL-ADM-02047:
Could not send SISNAPI
message
GenericLog      GenericError    1       2006-02-07
10:12:58     (sacmd.cpp 10(38
8) err=902049 sys=0) SBL-ADM-02049:
There is no connected server targeted for th
at
command
GenericLog      GenericError    1       2006-02-07
10:12:58     (sacmdl.cpp 29(6
71) err=902049 sys=0) SBL-ADM-02049:
There is no connected server targeted for t
hat command
$


(to be continued)






Cause


Configuration/ Setup


Solution



Message 1


(continuing...)



The enterprise log file shows:

...

ServerLog       ProcessExit     1       2005-12-07 14:31:27     S3SuspectAssigne

d46085     SBL-OSD-02010   Process exited with error - Process exited because of

a bus error (data alignment) SIGBUS

ServerLog       ProcessExit     1       2005-12-07 14:31:27     S3RouteNational

46090     SBL-OSD-02010   Process exited with error - Process exited because of

a bus error (data alignment) SIGBUS

ServerLog       ProcessExit     1       2005-12-07 14:31:27     IBSNCSTM

46088     SBL-OSD-02010   Process exited with error - Process exited because of

a bus error (data alignment) SIGBUS

ServerLog       ProcessExit     1       2005-12-07 14:31:28     IBSNEscalate

46087     SBL-OSD-02010   Process exited with error - Process exited because of

a bus error (data alignment) SIGBUS

...





(to be continued)


Message 2


(continuing ...)



The PSTACK of the coredump shows:

eagnmnsu257(ROOT)# pstack -F core.global.sun4u.eagnmnsu257.1138992838.5252.900.siebsess.7025

core 'core.global.sun4u.eagnmnsu257.1138992838.5252.900.siebsess.7025'
of 7025: siebsess 44 1
/u00/siebel/siebsrvr/admin/s3_dev_es.s3_dev_s1.shm 263 e

----------------- lwp# 1 / thread# 1 --------------------

b98c538c __1cJqeLicPath6FpkCpC_v_ (62656c2f, 62656c2f, 73696562, 73727672, 2f6d772f, 6c69623a) + 2c

----------------- lwp# 2 / thread# 2 --------------------

bfb1f090 _signotifywait (bf02c000, bfbe0b80, 0, bf6c0ef8, 2f, 7efefeff) + 8

bfb1a534 thr_errnop (0, 0, 0, 0, 0, 0) + 20

----------------- lwp# 3 --------------------------------

bf019300 private___lwp_cond_wait (bed65d98, bf02cd74, bf02c000, 0, 0, 4) + 8

bfb1cc8c _door_return (bed65cd8, bf00a380, 0, 0, 0, 0) + 68

----------------- lwp# 4 --------------------------------

bfb1cc34 _door_return (25, bf02c000, bf02d678, 3, bf02c000, 1) + 10

bf00a380 _lwp_start (bea0bd98, 0, 0, 0, 0, 0) + 18

bfb1a534 thr_errnop (0, 0, 0, 0, 0, 0) + 20

-------------------------- thread# 3 --------------------

bf00d9e0 _reap_wait (bf030988, 1e8fc, 0, bf02c000, 0, 0) + 38

bf00d738 _reaper (bf02ce08, bf032710, bf030988, bf02cde0, 1, fe400000) + 38

bf01b11c _thread_start (0, 0, 0, 0, 0, 0) + 40



(to be continued)


Message 3


Output of a truss on the srvrmgr is attached to this SR for your review.



Also, when we ran the odbcsql, we also got a core dump, the pstack is the same as the pstack for srvrmgr.



we are able to start the siebel servers and the callcenter OM, and
access screens other than the server admin screen. But the OM's with
SIGBUS do not work any more.



Even more confusing is that we we cold start (reboot the machines), the
error does not appear. However, this is not a acceptable solution.
Because we are running on Solaris machines, reboot the production won't
be thinkable.



If you can assist us in find and resolve the problems before we move to production, it's great.



Thanks,


Message 4


(1/2)



For the benefit of other readers:



It was found in the truss output and in the siebsrvr_xx.log logs located
in siebsrvr/log directory the error SBL-SVR-00027 error message found.
The error may be due to lack of file descriptors.



Based on above, the customer was recommended to follow the instructions
on the bookshelf below and adjust the kernel settings appropriately.



Siebel Server Installation Guide for UNIX > Tuning UNIX Operating
Systems for Siebel Installation > Tuning Siebel eBusiness
Applications for Solaris > Tuning Kernel Settings for Solaris



The kernel adjustments alleviated the problem for a while, but the SIGBUS error and core dumps start happening again.



Please, see below the findings that I have found about in the truss.out
file provided, within this file there is the _LD_LIBRARY_PATH value
used, as you can see below there is no reference for the
/u00/oracle/product/920/v64/lib32 directory, also it is repeating some
path directories some times. This is exactly what I would like to avoid
in the tests that I have asked you.



The LD_LIBRARY_PATH was reviewed and it was found a reference to the
Oracle 64 bit library directory. The customer was asked to remove the
reference to the Oracle 64 bit library keeping only the Oracle 32 bit
library as it is the only supported version of Oracle client libraries.



After removing the Oracle 64 bit library path from the LD_LIBRARY_PATH the SIGBUS error and core dumps did not appear anymore.


Message 5


(2/2)



The following is found in the Bookshelf > SIEBEL SERVER INSTALLATION
GUIDE FOR UNIX VERSION 7.5.3 JULY 2003 12-FN585F > Installing the
Siebel Server.



NOTE: Siebel applications support the Oracle 32-bit client, but if you
have installed the Oracle 64-bit client on your Siebel Server, you need
to add $ORACLE_HOME/lib32 instead of $ORACLE_HOME/lib to your
LD_LIBRARY_PATH (Solaris),SHLIB_PATH (HP-UX), or LIBPATH (AIX).



Kind Regards,












Applies to:


Siebel System Software - Version 7.8.2.3 SIA [19221] and later
Oracle Solaris on SPARC (64-bit)

Product Release: V7 (Enterprise)

Version: 7.8.2.3 [19221] Com/Med

Database: Oracle 9.2.0.7

Application Server OS: Sun Solaris 9

Database Server OS: Sun Solaris 8



This document was previously published as Siebel SR 38-3360281273.







Symptoms


SBL-DAT-00803, SBL-JCA-00317, SBL-SMI-00126, SBL-NET-01034


Hi,



We are experiencing two unexpected behaviors when attempting to log into eCommunications:



1) The Login Page is not displayed. "Page Cannot Be Displayed" error after about 10 minutes.

2) The Login Page is displayed, but "Page Cannot Be Displayed" error after attempting to log in.



We are able to access the database via a Dedicated Client and can use the ServerManager commands.

We have already tried bypassing all Load Balancers and the LDAP Server unsuccessfully.

We are currently getting JCA Connection errors from Portal to Siebel.

We are seeing excessive CPU for Kernal and IOWait.



We are seeing some discrepancies with the mounted file system.

After removing the Anonymous User *.spf file from the userpref directory, we started getting the Home Page.

However, the system attempted to create the new file but it has 0 byte,
and the file name convention is different, for example,
<username>&<application>.spf_<hostname>_14084_13.



This seems to be related to file access rights, but we made everything 777 and still get the same error.

We know that this UNIX machine had some problems with this mount point after a power outage, but we do not know the details.

We turned up logging for eComm and SCBroker and the logs stop just prior to getting access the UserPref file.



For eCustomer, it looks like we can connect via JCA for the initial
request (GetMDN), but then for the GetHierarchy we get an error and lose
the JCA connection:



[SIEBEL ERROR] [2007-05-25 12:25:55.892] [SiebelConnection(1230636829)]
Complex Account Hierarchy - Portal Services.GetHierarchyByUser FAILED on
Connection
(siebel.tcpip.none.none://<hostname>:2321/esblr/eCustomerCMEObjMgr_enu/!3.6e01.4893.46570dcc,
INT_PORTAL with message <com.siebel.om.sisnapi.RequestException>

<Error><ErrorCode>8236</ErrorCode> <ErrMsg>OMRPC
Request 4 on connection 18579 was abandoned after 60070 ms because it
timed out. (SBL-JCA-317) </ErrMsg></Error>

</com.siebel.om.sisnapi.RequestException>



Thanks!



Cause


Configuration/ Setup



Solution



Message 1


For the benefit of other readers:



Users are getting “This page cannot be displayed – Cannot find server or
DNS error” error message when trying to load the login page or after
providing username and password.



After deleting the User Preferences File, this behavior was resolved,
but the new user preference file was created with zero byte.

We have ruled out the possibility of this behavior being related to the
situation described in Alert 1025: Siebel Server Components That Read or
Write to Files on Solaris May Encounter Fopen() Behaviors.

We have checked all Operating System user limits.



After further investigation, customer found out that a lockd daemon
process was not running on the server where the shared Siebel File
System resides after the power outage.

All the other Siebel Servers have NFS mounts pointing to this server to access the shared File System.

Once lockd was started on that server, the behavior was no longer observed.



The following Service Requests helped troubleshooting this behavior:



    - Doc ID 504354.1: Unable to connect to Public Sector with SisnSockError

    - Doc ID 495979.1: Web client hangs when trying to login



Thank you,










Applies to:


Siebel System Software - Version: 7.8.2.2 [19219] and later   [Release: V7 and later ]
z*OBSOLETE: Microsoft Windows Server 2003

Product Release: V7 (Enterprise)

Version: 7.8.2.2 [19219]

Database: Oracle 9i

Application Server OS: Microsoft Windows 2003 Server SP1

Database Server OS: Sun Solaris 9



This document was previously published as Siebel SR 38-3035052921.



Symptoms


SBL-SVR-01015, SBL-SVR-01043, SBL-SMI-00033, SBL-NET-01034, SBL-NET-01033, SBL-ADM-01044, SBL-SCB-00015, SBL-SCB-00011Hi,

Each night a cold DB backup is taken on each environment. This is affecting a number
of components on the Siebel server:

Transaction Processor
Transaction
Merger
Transaction Router
Server Tables Cleanup

These components need to be
restarted each morning as the backup is affecting them. I decided to use scripts to shutdown the
siebel server 15 mins before backup is taken and another to restart it in the morning. I have
tested these script on one environment and this works fine. On another environmnet the shutdown
script appears to hang (i.e batch job does not complete). All the parameters are correct as the
command would not run otherwise. I have attached the script and the log files at the time it
executed (01:45). As you can see from the log file at 06:00 created after the startup script is
executed, the server is still shutting down. Can you please inform me what the issue
is?

Kind regards






Cause


Configuration/ Setup


Solution



Message 1


For the benefit of other readers:



The customer's script:



D:\sea78\test\siebsrvr\BIN\srvrmgr /g UKGateway /e siebtest /s SiebTest1 /u sadmin /p password /c "shutdown appserver SiebTest1"



In review of the customer's component log files, we could see that there
was a problem with one or more components shutting down. As stated in
SupportWeb SR 38-1147657941 [Siebel Servers stick in "Shutting down"
state], the "shutdown appserver" command issued from the Server manager
command-line is a synchronous process, thus the command will block until
all tasks in the server have exited. There could be a problem stopping
one or more components, caused either by the components themselves, or
caused by the synchronous nature of the shutdown process (ie. one
component is waiting on a response from another component, or the Siebel
server - that may have already been shutdown).



The customer was recommended to use the Windows command, net stop. The net stop command is used as follows:



net stop <service>



For example:



net stop SiebSrvr



Which would stop the service called SiebSrvr.



Assuming you do not have any problem stopping your Siebel Server service
using the Windows' Services utility, then you should not experience a
problem with using the net stop command to stop the Siebel Server
service. You can schedule the command using the Windows command, at.



This information resolved this for the customer.



Regards,








Applies to:


Siebel System Software - Version 7.7.1 [18306] to 8.2.2.1 SIA[23012] [Release V7 to V8]
Information in this document applies to any platform.

Error Message Area:Networking Layer - NET

Version:Siebel 7.7







Purpose


This document is intended to provide cause and corrective action
information about Siebel Error Message SBL-NET-01034: The SISNAPI
connection was closed by the peer.



Scope


This document is informational and intended for any user.



Details



Explanation


The connection to the server has been explicitly closed by the
client. This may be due to server or client process termination. This
error could also occur in any Siebel Server component such as MQ or
Remote Synchronization, due to incorrect installation or a time out
condition.



Corrective Action


Investigate whether this error occurs for multiple clients on
different machines or for multiple server components. Determine the
steps that led up to the error and determine if a long period of waiting
or network interruption coincides with the error.



Depending on the cause, verify the installation requirements or verify network connectivity and re-attempt the operation.










Applies to:


Siebel Remote - Version: 7.5.3.4 [16180] to 8.1 [21039] - Release: V7 to V8
IBM AIX on POWER Systems (64-bit)

Product Release: V7 (Enterprise)

Version: 7.5.3.4 [16180]

Database: Oracle 8.1.7.4

Application Server OS: IBM AIX 5L 5.1

Database Server OS: IBM AIX 5L 5.1



This document was previously published as Siebel SR 38-1923792601.



Symptoms


SBL-GDB-00004, SBL-SMI-00024, SBL-SRB-00061, SBL-GEN-09103Hi,

I am running "GenNewDb" and immediately, I got this following
error.

srvrmgr:nytids007tst> start task for component GenNewDb

start task for
component GenNewDb
SBL-NET-01034: The SISNAPI connection was closed by the peer

When I
checked the "GenNewDb_xxxx.log",I can see the following error
message.

ContextInit     ContextInit     0       2005-04-18
23:15:50     SIEBEL_ASSERT_MODE =
0
GenericLog      GenericWarn     2       2005-04-18
23:15:50     (smisched.cpp 17(266) err=9103 sys=0) SBL-GEN-09103:
Parameter value
was never set (i.e. is
null)
GenericLog      GenericWarn     2       2005-04-18
23:15:50     (smisched.cpp 17(269) err=9103 sys=0) SBL-GEN-09103:
Parameter value
was never set (i.e. is
null)
GenericLog      GenericError    1       2005-04-18
23:15:50     (smisched.cpp 17(285) err=2100024 sys=0) SBL-SMI-00024:
Unable to lo
ad gennewdb

Please look into this ASAP as we need to provide the extract
to TAM for review.

Thanks & Regards






Cause


Missing siebel server files

a] \siebsrvr\lib\libgennewdb.so
b] \siebsrvr\dbtempl\sse_utf8.dbf


Solution



Message 1


1 of 2



For the benefit of other readers, the issue and resolution are as follows.



Issue Description:

----------------------

When the GenNewDB was run, wither from the UI or from server manager, an
error occurred alomost immediately. In this case, customer ran from the
server manager prompt as follows. Customer encountered errors as
described below.



srvrmgr:nytids007tst> start task for component GenNewDb



start task for component GenNewDb

SBL-NET-01034: The SISNAPI connection was closed by the peer



When I checked the "GenNewDb_xxxx.log",I can see the following error message.



ContextInit     ContextInit     0       2005-04-18 23:15:50     SIEBEL_ASSERT_MODE = 0

GenericLog      GenericWarn     2       2005-04-18
23:15:50     (smisched.cpp 17(266) err=9103 sys=0) SBL-GEN-09103:
Parameter value

was never set (i.e. is null)

GenericLog      GenericWarn     2       2005-04-18
23:15:50     (smisched.cpp 17(269) err=9103 sys=0) SBL-GEN-09103:
Parameter value

was never set (i.e. is null)

GenericLog      GenericError    1       2005-04-18
23:15:50     (smisched.cpp 17(285) err=2100024 sys=0) SBL-SMI-00024:
Unable to lo

ad gennewdb



Resolution:

--------------



After detailed analysis, Siebel Technical Support discovered the following.



1. Customer environment was missing the following files



a] \siebsrvr\lib\libgennewdb.so

b] \siebsrvr\dbtempl\sse_utf8.dbf



Contd...


Message 2


2 of 2 - Contd from previous..



Customer had another environment similar to this one, where everything
was working. Since this non working environment was a test environment
and not a production environment, Siebel Technical Support recommended
copying the missing libray file \siebsrvr\lib\libgennewdb.so and
\siebsrvr\DBTEMPL\sse_utf8.dbf from the working environment.



After the copy was done, customer restarted the siebel server and gateway and ran the GenNewDB and it worked successfully.



Recommendations:

-----------------------



Other recommendations made to the customer were as follows



1. Refer Alert 899: Maintenance Level Update Recommended for AIX 5L v5.1



According to the Alert 899, which refers Alert 851, Customers have to
ensure Siebel deployment is running the latest C++ runtime libraries
(6.0.0.12) or later, if they are on AIX 5L v5.1



2. Other recommendation made was to not copy missing files, if the
environment is production. If missing files are noticed in production,
then for long term stability, customers must consider reinstallign their
Siebel environment.



Thank you



















תגובות

פוסטים פופולריים מהבלוג הזה

FINS Data Transfer Utilities

SBL-BPR-00191: The rowId of the active row of the primary buscomp '%1', '%2', does not match the Primary Id

Profile Attributes and Open UI