Hi,
After some hours of operation, the (virtual) memory usage grows up to 50GB and more (and keeps growing and growing). While RX and TX audio keep working without interruptions, the UI reaction gets noticeably slower. Physical memory usage stays at ~3.5GB.
I noticed this for a few times, but as it does not happen each time I launch the console, I currently cannot describe a way to reproduce the issue.
73 Jean DJ0VL
3.0.25 - high memory usage
- Simon G4ELI
- Posts: 2134
- Joined: Thu Aug 06, 2020 7:27 am
- Location: Mawnan Smith
- Contact:
Re: 3.0.25 - high memory usage
Is this only when using Pluto via LAN?
If you see this again please:
If you see this again please:
- Add a logfile.
- Make a note of how long the radio has been running and the virtual memory. From time and memory I can maybe work out where this could be happening.
Re: 3.0.25 - high memory usage
Hello Simon,
Pluto via LAN is my only current use case, so I can't tell if this happens with other configurations as well. I forgot to attach the logfile, here it comes. Virtual memory usage was at 50GB after some 9 hours runtime.
From memory, the console was running for 2 or 3 hours before I noticed the increasing memory usage. At that time, the current total memory usage dsiplay in the options window increased by ~1MByte/sec.
I will continue to observe and make notes as you asked. What comes to mind is that log4om was running in parallel and I assume there is a kind of access competition to the GPU when both programs are running. There is a slight improvement in the response time of log4om when I disable CUDA usage in SDR Console. I will observe this as well.
73 Jean DJ0VL
Pluto via LAN is my only current use case, so I can't tell if this happens with other configurations as well. I forgot to attach the logfile, here it comes. Virtual memory usage was at 50GB after some 9 hours runtime.
From memory, the console was running for 2 or 3 hours before I noticed the increasing memory usage. At that time, the current total memory usage dsiplay in the options window increased by ~1MByte/sec.
I will continue to observe and make notes as you asked. What comes to mind is that log4om was running in parallel and I assume there is a kind of access competition to the GPU when both programs are running. There is a slight improvement in the response time of log4om when I disable CUDA usage in SDR Console. I will observe this as well.
73 Jean DJ0VL
- Attachments
-
- SDR Console tmp.txt
- (425.92 KiB) Downloaded 29 times
- Simon G4ELI
- Posts: 2134
- Joined: Thu Aug 06, 2020 7:27 am
- Location: Mawnan Smith
- Contact:
Re: 3.0.25 - high memory usage
Thanks, it's losing ~ 1.5 MB/second.
Re: 3.0.25 - high memory usage
Hello Simon,
Today, the issue showed up again. Console was running with Pluto via LAN for about 2 hours with memory usage below 1GB, LOG4OM was running in parallel. After switching back from LOG4OM to SDR-Console, I noticed a change in the spectrum/waterfall display. The noise level showed up around 5 dB higher than before switching to the other program. As I already observed this before, I stopped the radio, restarted it and the spectrum display was at normal level again.
After restarting the radio, the memory usage started to increase (see screenshots in the attached PDF). About 2 hours later, console crashed due to out-of-virtual-memory. Memory usage was around 20GB when the crash occured, the system limit was reached as I did also run Process Exporer and Process Viewer, hoping to get some additional information about the process/dll that graps the memory. Unfortunately, I could not identify a dll taking all of the memory.
Debug View was running as well, the output is attached. As the crash was occurring, lots of buffer overflows were recorded in the system log. There is probably nothing worth to detect from the crashlog apart from the out-of memory, I'm nevertheless uploading the zip to a google-drive and will send you a link after posting this reply.
73 Jean DJ0VL
Today, the issue showed up again. Console was running with Pluto via LAN for about 2 hours with memory usage below 1GB, LOG4OM was running in parallel. After switching back from LOG4OM to SDR-Console, I noticed a change in the spectrum/waterfall display. The noise level showed up around 5 dB higher than before switching to the other program. As I already observed this before, I stopped the radio, restarted it and the spectrum display was at normal level again.
After restarting the radio, the memory usage started to increase (see screenshots in the attached PDF). About 2 hours later, console crashed due to out-of-virtual-memory. Memory usage was around 20GB when the crash occured, the system limit was reached as I did also run Process Exporer and Process Viewer, hoping to get some additional information about the process/dll that graps the memory. Unfortunately, I could not identify a dll taking all of the memory.
Debug View was running as well, the output is attached. As the crash was occurring, lots of buffer overflows were recorded in the system log. There is probably nothing worth to detect from the crashlog apart from the out-of memory, I'm nevertheless uploading the zip to a google-drive and will send you a link after posting this reply.
73 Jean DJ0VL
- Attachments
-
- SDR Console Memory Usage 20202002.pdf
- (751.63 KiB) Downloaded 28 times
-
- SystemDebugLog 20201002.txt
- (293.26 KiB) Downloaded 19 times
- Simon G4ELI
- Posts: 2134
- Joined: Thu Aug 06, 2020 7:27 am
- Location: Mawnan Smith
- Contact:
Re: 3.0.25 - high memory usage
While memory is increasing can you actually use the radio, tune and demodulate?
Thanks for the Debug log, very interesting.
Thanks for the Debug log, very interesting.
Re: 3.0.25 - high memory usage
Yes, the radio modulation/demodulation is usable all time in full duplex, there is no noticeable interruption on RX or TX when memory usage is increasing. Radio continued working even while the waterfall freezed for several seconds after switching back from LOG4OM. I'm not sure if tuning was possible while the waterfall was freezed, it was possible for sure while the memory usage increased.
Meanwhile, I suspect some strange interaction with LOG4OM, maybe some access conflict to the GPU? After the crash this afternoon, I did terminate LOG4OM and SDR console is now running for around 5 hours without problems, memory usage stays at < 1GB. Maybe I need to look for an alternative log program
73 Jean DJ0VL
Meanwhile, I suspect some strange interaction with LOG4OM, maybe some access conflict to the GPU? After the crash this afternoon, I did terminate LOG4OM and SDR console is now running for around 5 hours without problems, memory usage stays at < 1GB. Maybe I need to look for an alternative log program
73 Jean DJ0VL
- Simon G4ELI
- Posts: 2134
- Joined: Thu Aug 06, 2020 7:27 am
- Location: Mawnan Smith
- Contact:
Re: 3.0.25 - high memory usage
No, let me work on this.
- Simon G4ELI
- Posts: 2134
- Joined: Thu Aug 06, 2020 7:27 am
- Location: Mawnan Smith
- Contact:
Re: 3.0.25 - high memory usage
So,Jean wrote: ↑Fri Oct 02, 2020 7:37 pm Yes, the radio modulation/demodulation is usable all time in full duplex, there is no noticeable interruption on RX or TX when memory usage is increasing. Radio continued working even while the waterfall freezed for several seconds after switching back from LOG4OM. I'm not sure if tuning was possible while the waterfall was freezed, it was possible for sure while the memory usage increased.
Meanwhile, I suspect some strange interaction with LOG4OM, maybe some access conflict to the GPU? After the crash this afternoon, I did terminate LOG4OM and SDR console is now running for around 5 hours without problems, memory usage stays at < 1GB. Maybe I need to look for an alternative log program
73 Jean DJ0VL
I've been through the logic and applied more sanity checks in areas where processing queues could build up. I have checks to make then will post in the Test Team sub forums where you are a member.
It's not LOG4OM that's the problem, worst case is that LOG4OM has exposed a coding oversight in SDR Console.