Professional OPC
Development Tools

logos

Online Forums

Technical support is provided through Support Forums below. Anybody can view them; you need to Register/Login to our site (see links in upper right corner) in order to Post questions. You do not have to be a licensed user of our product.

Please read Rules for forum posts before reporting your issue or asking a question. OPC Labs team is actively monitoring the forums, and replies as soon as possible. Various technical information can also be found in our Knowledge Base. For your convenience, we have also assembled a Frequently Asked Questions page.

Do not use the Contact page for technical issues.

Application Hangs as a result of a triggered Garbage Collection

More
22 Oct 2020 12:53 #9020 by support
Hello,

I do not quite understand the customer's response. You wrote that they have not tried to disable prerequisites boxing. We have not proposed anything else. So, have they changed anything? If not, why are they reporting that the same thing has happened again? What new information is in it?

Regarding the remaining part of the response: I think it is not likely that the use of InvokeRequired/Invoke is related to the issue - but of course, it makes no harm to try a different approach. For more information related to this, they can refer to: opclabs.doc-that.com/files/onlinedocs/QuickOpc/Latest/User%2...g%20and%20Synchronization.html .

Best regards

Please Log in or Create an account to join the conversation.

More
21 Oct 2020 13:31 #9019 by gkalipershad
Hello,

Customer responds:

"The application did manage to run for 4 days, then hung again. CPU usage is zero when this happens. I suspect it’s something due to Subscriptions as it happens frequently with a test environment where we have subscribed to 62 tags, but happens much less frequently on a system with only 3 tags. I suspect it may be related to the way in which the events are passed through to the calling application, if those are not being raised on the thread that creates the OPC client object. I will be looking at this again later today, part of this testing may involve changing the way that the change data events are handled, at present they use InvokeRequired and Invoke to perform UI updates but we may try to instead update a thread safe ConcurrentDictionary and then use a timer in the UI thread to update the UI data values from this dictionary object instead, thereby disconnecting the OPC client event thread from the UI thread."

They have not yet tried to disable prerequisites boxing. What he is going to try and change about his application is kind of going above my head, but I will let you know what he says the result of that is.

Regards,
GK

Please Log in or Create an account to join the conversation.

More
12 Oct 2020 12:35 #8992 by gkalipershad
Hello,

Thanks for the additional information. I will recommend disabling the prerequisites boxing and will check with the customer on your other question.

Regards,
GK

Please Log in or Create an account to join the conversation.

More
10 Oct 2020 09:05 #8989 by support
Hi G.,
thanks for more details.

First, some comment to the last paragraphs of the customer's reply, because I think there may be some misunderstanding of how the GC works.

In my understanding, it is perfectly OK and normal that:
- GC is is invoked from time to time by the .NET CLR, even without the code actually calling it. It happens at non-deterministic time and any thread can "trigger" it - also that cannot be predicted.
- While GC does its work, it suspends all other managed threads in the process (more precisely, in the .NET application domain).

So all that is fine. What is not OK is that the GC seems to hang.

That is all Microsoft code and we have almost no influence in it. I suggest that the customer tries to disable the prerequisites boxing (kb.opclabs.com/How_to_disable_prerequisites_boxing ). I know that seems to be quite common recommendation from me nowadays, but the library involved (BoxedApp SDK) does some hackish low-level things to the system, so I can imagine it being a cause for various problems (older versions of it have proven problems, but they usually manifested as a crash; but you never know...). Disabling it puts the software into a "normal" state in which the described "hang" really should not happen.

One more question/verification: Is the CPU usage % of the process at/near zero when it hangs, or does it hang "live" - with a non-zero CPU usage % indicating probably that the GC thread might be caught in an infinite loop?

Regards

Please Log in or Create an account to join the conversation.

More
09 Oct 2020 12:41 #8986 by gkalipershad
Hello,

The customer replies:

"The application hangs – it doesn’t respond to anything, and has to be terminated using Task Manager. It’s as if the UI thread is stuck, which according to the DebugDiag2 output it is, as it’s waiting for the garbage collection in the OPC UA component thread to finish, but it never does. It’s not a crash – the DebugDiag2 analysis of a memory dump is for both crashes and hangs, hence the folder name, that’s a Microsoft convention and not something we defined.

The version number of the OPC UA DLL is, according to Visual Studio, 5.58.257.1.

We’ve also had the same issue with 2018.3 (version 5.54.453.1), and 2017.1 (version 5.50.323.1). We’re in the process of moving from a VB6 OPC DA ActiveX based application to a .NET version using UA, hence why we have multiple versions.

It does happen every time, but it can take hours or days for it to happen. It’s “random” in that we can’t predict how long it will take for it to happen, but it seems to happen quicker when we have more subscriptions, and every time we’ve run DebugDiag2 on a memory dump of the hung process, we’ve seen the same thing where a number of threads, including the UI thread 0, are hung waiting for the GC to complete, which has been triggered inside the thread performing an OPC UA Write.

What’s of concern is that if the data changed events are not being raised on the thread that created the OPC UA client then this could cause a hang, it might be that the GC is a bit of a red herring in that the hang occurs and it just happens that there is a GC happening somewhere. I’ve tried wrapping the event handlers in the application with checks to see if Invoke is required and not getting that, and the hang is happening before the event handler code starts – I’ve added simple logging to the start and end of all function calls and the hang occurs outside of any of the main application functions and event handlers."

Please let me know if you need anything else or require any further clarification.

Thanks,
GK

Please Log in or Create an account to join the conversation.

More
08 Oct 2020 19:43 #8985 by support
Hello,

1. I need some clarification please: Regarding "...application is hanging". Does it mean that it pauses for a while and then continues running, or does it hang for good? Why is the report named CrashHangAnalysis - is there any crash involved?

2. Which build of 2020.2 is used?

3. Does this happen always in their program, or randomly how frequently?), or is there any pattern observed?

Some notes before we go further:

I am not completely sure how to interpret the message "Thread 32 triggered the garbage collection." in the analysis report. I do not think that it means that the GC was invoked explicitly by the thread. It can mean that it was invoked by .NET simply because .NET does it from time to time, non-deterministically. Similarly to the user's code, our own code does not explicitly invoke GC anywhere, and as far as I can tell, neither does the OPC UA stack invoke it. (but even if somebody did, by itself it should not cause a hang).

"...not sure where the other threads are being spun up". This is normal. Threads are created for various tasks inside the QuickOPC code, and in many cases, threads from a thread pool are used.

Best regards

Please Log in or Create an account to join the conversation.

More
08 Oct 2020 16:57 #8984 by gkalipershad
Hello,

I am working with a customer with the following setup:

Toolkit Version: 2020.2
Development Environment: Visual Studio 2019
Development Language: VB.NET
OPC Standard Used: OPC UA
.NET Framework Version Targeted: 4.7.2
Operating System: Issue is present on Win 10 (1907 & 1609), Server 2012R2, Server 2016, Server 2019

The customer's .NET application is hanging. I have attached a memory dump analysis that they sent me where we are seeing that a garbage collection, which is triggered by a thread in the OPC client, is preventing the application from proceeding.

At no point in their code are they manually triggering a GC.

The GC is triggered during a call to Opc.Ua.Bindings.TcpMessageSocket.WriteMessage. This is a single threaded application, so I am not sure where the other threads are being spun up. Is there a way to prevent this GC from occurring or some other way to prevent this hang based on what you see?

Please let me know what additional information I may need to obtain.

Thanks,
GK
Attachments:

Please Log in or Create an account to join the conversation.

Moderators: support
Time to create page: 0.097 seconds