Exceptions during stress test

[Burkhard]

Hello

I'm working as an external developer for the Munich Re.
We currently doing stress testing on our Application. We are using Telerik.OpenAccess for .Net 2.0 version 2009.1.405.1. 
During two stress tests we got the exceptions below. We never saw these exceptions outside the stress test.
Could you please explain the exceptions below and tell me, how I can avoid them?

Regards,
    Burkhard
------------------------------

  <Timestamp>7/9/2010 2:13:04 PM</Timestamp>

<Message>Exception occured in WCF service

Telerik.OpenAccess.OpenAccessException: Instance not managed: MunichRe.LRCF.BusinessObjects.AggregationSource@2be751d

   at Telerik.OpenAccess.RT.ExceptionWrapper.Throw()

   at OpenAccessRuntime.DataObjects.OpenAccessPersistenceManagerImp.getInternalSM(PersistenceCapable pc)

   at OpenAccessRuntime.DataObjects.PMProxy.fillCopy(OfflineStateManager cpy, PersistenceCapable pc)

   at Telerik.OpenAccess.ObjectContainer.FillFrom(IEnumerator roots, IPersistenceManager from, IEnumerator allobjects, String listName)

   at Telerik.OpenAccess.ObjectContainer.CopyFrom(IObjectScope objectScope, String listName, Object objectToBeAdded, IObjectCollector objectCollector)

   at MunichRe.LRCF.Server.DataAccess.ObjectContainerQueries.GetDataFolders(IObjectScope scope, List`1 periodIds, Nullable`1 riskModelSector, ObjectContainer container)

   at MunichRe.LRCF.Server.DataService.RequestHandlers.GetDataFoldersRequestHandler.OnHandleRequest(ObjectContainer container, GetDataFolderRequest request)

   at MunichRe.LRCF.Server.DataService.DataRequestHandler`1.HandleRequest(ObjectContainer container, GetDataRequest request)

   at MunichRe.LRCF.Server.DataService.DataServiceImpl.<>c__DisplayClass1.<GetData>b__0(GetDataRequest request)

   at MunichRe.LRCF.Utilities.EnumerableExtensions.Each[TItem](IEnumerable`1 collection, Action`1 action)

   at MunichRe.LRCF.Server.DataService.DataServiceImpl.GetData(List`1 requests)

   at SyncInvokeGetData(Object , Object[] , Object[] )

   at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)

   at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)

   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)

   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(MessageRpc& rpc)

   at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)</Message>

<Category>Trace, LRCFServerLogging, Foundation</Category>

<Priority>2</Priority>

<EventId>0</EventId>

<Severity>Error</Severity>

<Title></Title>

<Machine>MUCS800365</Machine>

<ApplicationDomain>MunichRe.LRCF.Server.exe</ApplicationDomain>

<ProcessId>8936</ProcessId>

<ProcessName>P:\Program Files\Munich Re Group\MunichRe LRCF Server\MunichRe.LRCF.Server.exe</ProcessName>

<Win32ThreadId>6516</Win32ThreadId>

<ThreadName>LoggingThread</ThreadName>

<ExtendedProperties></ExtendedProperties>

------------------------------------------------------------------------------------

<Timestamp>7/9/2010 2:34:05 PM</Timestamp>

<Message>Exception occured in WCF service

System.Exception: Intentionally not implemented.

   at Telerik.OpenAccess.RT.CopySM.getObjectId(PersistenceCapable pc)

   at MunichRe.LRCF.BusinessObjects.CalculationSource.OpenAccessEnhancedGetObjectId()

   at Telerik.OpenAccess.ObjectContainer.FillFrom(IEnumerator roots, IPersistenceManager from, IEnumerator allobjects, String listName)

   at Telerik.OpenAccess.ObjectContainer.CopyFrom(IObjectScope objectScope, String listName, Object objectToBeAdded, IObjectCollector objectCollector)

   at MunichRe.LRCF.Server.DataAccess.ObjectContainerQueries.GetDataFolders(IObjectScope scope, List`1 periodIds, Nullable`1 riskModelSector, ObjectContainer container)

   at MunichRe.LRCF.Server.DataService.RequestHandlers.GetDataFoldersRequestHandler.OnHandleRequest(ObjectContainer container, GetDataFolderRequest request)

   at MunichRe.LRCF.Server.DataService.DataRequestHandler`1.HandleRequest(ObjectContainer container, GetDataRequest request)

   at MunichRe.LRCF.Server.DataService.DataServiceImpl.<>c__DisplayClass1.<GetData>b__0(GetDataRequest request)

   at MunichRe.LRCF.Utilities.EnumerableExtensions.Each[TItem](IEnumerable`1 collection, Action`1 action)

   at MunichRe.LRCF.Server.DataService.DataServiceImpl.GetData(List`1 requests)

   at SyncInvokeGetData(Object , Object[] , Object[] )

   at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)

   at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)

   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)

   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(MessageRpc& rpc)

   at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)</Message>

<Category>Trace, LRCFServerLogging, Foundation</Category>

<Priority>2</Priority>

<EventId>0</EventId>

<Severity>Error</Severity>

<Title></Title>

<Machine>MUCS800365</Machine>

<ApplicationDomain>MunichRe.LRCF.Server.exe</ApplicationDomain>

<ProcessId>8936</ProcessId>

<ProcessName>P:\Program Files\Munich Re Group\MunichRe LRCF Server\MunichRe.LRCF.Server.exe</ProcessName>

<Win32ThreadId>6516</Win32ThreadId>

<ThreadName>LoggingThread</ThreadName>

<ExtendedProperties></ExtendedProperties>

 

 

[Jan Blessenohl]

Hi Burkhard,
Exactly at this place we have fixed something, but if this exception was only thrown if you register an tracing event and ask inside this event for the object id. This behavior was reproducable and should not pop up only in a stress test.

Do you know if the object that is throwing the exception (MunichRe.LRCF.BusinessObjects.CalculationSource) is a new object or just comming from the db?

Is there a chance to run the tests with a new OpenAccess version?

Regards,
Jan Blessenohl
the Telerik team

[Burkhard]

Hi Jan

thank you for your fast reply.

>Exactly at this place we have fixed something, but if this exception was only thrown if you register an tracing event and ask inside this >event for the object id. This behavior was reproducable and should not pop up only in a stress test.
BM> However we only have this Problem, when we are doing stress testing...
BM> Could the issue you have fixed explain both exceptions: "Instance not managed" and "Intentionally not implemented"?

>Do you know if the object that is throwing the exception (MunichRe.LRCF.BusinessObjects.CalculationSource) is a new object or just >comming from the db?
BM>It is NOT a new object: It is just coming from DB

>Is there a chance to run the tests with a new OpenAccess version?
BM> Could be a long term solution. We cannot switch version in short time. What version do you suggest? Probarbly the latest version...
Regards,
    Burkhard
 

[Jan Blessenohl]

Hello Burkhard,
Oh, sorry, though the second is the internal exception.

The first one looks like the scope has already been disposed. Are you using the same scope in multiple threads? This can also explain the second problem as well but i am not 100% sure.


Sincerely yours,
Jan Blessenohl
the Telerik team

[Burkhard]

Hi Jan

>The first one looks like the scope has already been disposed. Are you using the same scope in multiple threads? This can also >explain the second problem as well but i am not 100% sure.

Each (WCF) Server Request gets an extra scope. So we avoid using same scope in multiple threads.

As this occurres only in our stress test scenario (which consists only of read operations) a synchronization of the queries to object scope might fix this issue. I would not like this scenario at all, as we would loose reading parallel from the DB. Does open access supports synchronization? What do you think about this idea?

Regards,
    Burkhard

[IT-Als]

Hi Burkhard,

First, sorry for interfering your thread, but I think this is most definitely the handling of object scope that there's something about.
You're right the the best practices when speaking WCF is that when your request starts you should get a new object scope and your should dispose just before finishing your request.

During one request you must make sure that whenever you ask for a object scope you get the same that was instantiated when the request started. This means that you have to store the object scope somewhere once retrieved upon starting the request.

I have written a blog post on scope handling in WCF scenarios. Maybe you should check it out.

Regards

Henrik

[Jan Blessenohl]

Hello Henrik,
Means in your production environment you have one scope per thread but not in your stress test?

You can set the scope to multithreadded in the backend configuration but because this is far away from your real scenario it makes no sense to me. Can you change your stress test to use the same pattern as in production?

Sincerely yours,
Jan Blessenohl
the Telerik team

[IT-Als]

Hi Jan,

Was your answer/question destined for me?

Anyway, in our environments (both dev, test and prod) we use the one scope - one thread approach and it is working just fine.

We recently did a stress test on our WCF services and depending on your hosting (IIS, Windows Service, whatever) there's a lot of settings which can interfere how services perform... IIS recycling worker processes (which the WCF service runs under if it hosted by IIS) for one thing. Maybe this is an issue..

Regards

Henrik

[Jan Blessenohl]

Hello Burkhard,
did you manage to put some light into your issue?

@Henrik, sorry, was too fast with my answer, thanks for your know how.

Kind regards,
Jan Blessenohl
the Telerik team

[Burkhard]

Unfortunately not.
We just have decided to care about this issue later.
I will come back to this issue probarbly next week.

Regards
Burkhard
 

[Jan Blessenohl]

Hi Burkhard,
No problem, take your time.

Sincerely yours,
Jan Blessenohl
the Telerik team