Skip to main content
M
MichalTurek
Navigator | Tier 3
July 9, 2024
Question

On-Premise ControlRoom Server sudden insuficent disk space

  • July 9, 2024
  • 17 replies
  • 497 views

Hallo,

I have a Control Room on-premise (rev.33) installed on 500GB disk space VM and it got a sudden disk overusage and now its 0GB space of 499GB available.

The curious thing that checking all the folders on a disk get me a 68GB of disk usage.

I found in the Monitoring logs below inputs:

“[ERROR] 2024/07/09 08:46:14.746373 Error: File IO Fail 
Module: FileSystemLatencyMonitoring 
Method: getReadFileLatency 
Error Message: Error occured in Read file operation 
Underlying Error: EOF
[INFO] 2024/07/09 08:46:17.607182  ProcessMapUpdater : Execute : Updating process map
[ERROR] 2024/07/09 08:46:17.607714  ConfigChangeDetector : Execute : File .\config/cluster.properties doesn't exist
[ERROR] 2024/07/09 08:46:24.746745 Error: File IO Fail 
Module: FileSystemLatencyMonitoring 
Method: getWriteFileLatency 
Error Message: Error occured in Write file operation 
Underlying Error: write C:\ProgramData\AutomationAnywhere\Server Files/A360Monitoring/FileSystemLatency/muysRutT2024-07-09-08-46-24_Temp.txt: There is not enough space on the disk.
[ERROR] 2024/07/09 08:46:24.746967 Error: File IO Fail 
Module: FileSystemLatencyMonitoring 
Method: getReadFileLatency 
Error Message: Error occured in Read file operation 
Underlying Error: EOF
[ERROR] 2024/07/09 08:46:44.755509 Error: File IO Fail 
Module: FileSystemLatencyMonitoring 
Method: getWriteFileLatency 
Error Message: Error occured in Write file operation 
Underlying Error: write C:\ProgramData\AutomationAnywhere\Server Files/A360Monitoring/FileSystemLatency/OoEzqwAt2024-07-09-08-46-44_Temp.txt: There is not enough space on the disk.
[ERROR] 2024/07/09 08:46:44.755509 Error: File IO Fail 
Module: FileSystemLatencyMonitoring 
Method: getReadFileLatency 
Error Message: Error occured in Read file operation 
Underlying Error: EOF
[ERROR] 2024/07/09 08:46:47.604181  ConfigChangeDetector : Execute : File .\config/cluster.properties doesn't exist”

Is there a possibility thart CR server files reserve a space for a temp files or something?

Or is it some VM settings to look for?

looking for clues,

greeting Michal

17 replies

Show previous replies
M
MichalTurekAuthor
Navigator | Tier 3
October 1, 2024

Hello,

 

I faced the exact same error. I got a “Bad Gateway” error and found out about 120 GB of new relic .jar files.

Is there a solution to this problem ? I have deleted files and restarted the control room process, then restarted the virtual machine but the problem is still there

Hallo @Augustin 

for me turning off the services as mentioned above helped, but I still haven’t found out what is the real matter of this issue,

greeings

For me, as soon as I turn on the main service (“Automation Anywhere Control Room Service”) the files start being generated. And even if I turn it on knowing that files are being generated I still have a “Bad Gateway” error

@Augustin do You have everything with licensing sorted out properly? When have problems begun?

Has everything been working correctly up to now? Did You switch licensing?

A
Augustin
Flight Specialist | Tier 4
October 1, 2024

Hello,

 

I faced the exact same error. I got a “Bad Gateway” error and found out about 120 GB of new relic .jar files.

Is there a solution to this problem ? I have deleted files and restarted the control room process, then restarted the virtual machine but the problem is still there

Hallo @Augustin 

for me turning off the services as mentioned above helped, but I still haven’t found out what is the real matter of this issue,

greeings

For me, as soon as I turn on the main service (“Automation Anywhere Control Room Service”) the files start being generated. And even if I turn it on knowing that files are being generated I still have a “Bad Gateway” error

@Augustin do You have everything with licensing sorted out properly? When have problems begun?

Has everything been working correctly up to now? Did You switch licensing?

There is no licensing problem AFAIK (I am not responsible for the licenses in my job, but it was far from expiring when I last checked).

I have recently started using AARI (a few weeks ago) and have done a lot of testing yesterday. Given the speed at which the c:\ drive is filling up, the trigger of that problem can’t have been before the week-end.

A
Augustin
Flight Specialist | Tier 4
October 1, 2024

I can safely start the processes elasticsearch, compiler, caching, discovery (x 2), reverse proxy, messaging, storage service!-->

However, starting the processes main (the Control Room Service), iqbot & robotic interface service, it starts filling up the Temp folder with files!-->

M
MichalTurekAuthor
Navigator | Tier 3
October 1, 2024

I can safely start the processes elasticsearch, compiler, caching, discovery (x 2), reverse proxy, messaging, storage service!-->

However, starting the processes main (the Control Room Service), iqbot & robotic interface service, it starts filling up the Temp folder with files!-->

I’m affraid i won’t help. You should rise a ticket probably, let us know what was the problem.

 

greetings

A
Augustin
Flight Specialist | Tier 4
October 1, 2024

I have just read in the release notes of v.33 https://docs.automationanywhere.com/bundle/enterprise-v2019/page/v33-release-automation-workspace.html “ After you update to this release, when the .jar files are created by Newrelic in the Java TEMP folder, the older files are now cleared correctly. “

Is this the error we are talking about ?

Should I update from v.32 to v.33 ? (The control room is currently down, I absolutely would not like to upgrade in this condition)

M
MichalTurekAuthor
Navigator | Tier 3
October 1, 2024

I have just read in the release notes of v.33 https://docs.automationanywhere.com/bundle/enterprise-v2019/page/v33-release-automation-workspace.html “ After you update to this release, when the .jar files are created by Newrelic in the Java TEMP folder, the older files are now cleared correctly. “

Is this the error we are talking about ?

Should I update from v.32 to v.33 ? (The control room is currently down, I absolutely would not like to upgrade in this condition)

Yeah maybe thats the solution,

A
Augustin
Flight Specialist | Tier 4
October 1, 2024

After desperately searching for information I opened the file WebCR.log and found that the SQL password had expired. Changing the SQL password seemed to solve the error.

I do not know why that would generate a huge bunch of files and never notice me about that, but for now the problem seem corrected.

What a waste of time to not have useful error messages and notifications and having to search for it on our own, it is not the first time this kind of thing happens with AA.

Terms & ConditionsAccessibility statement