systemd journal misbehavior

Development on core packages and the distribution goes on in here.

systemd journal misbehavior

Postby shaman2014 » Tue Aug 26, 2014 5:04 am

I have been working on a project using a RaspPi in an embedded application that is running ArchLinux, the project has been working for 1.5 years so far with only a few issues. Recently we decided to use RabbitMQ to handle messaging between applications and a server apps. I have successfully installed the rabbitmq software and started the server with no problems and I have built several applications which are working great. Recently a new problem has developed which I have had no luck in determining what it going on. Anywhere from 30 secs to 30 mins after I start sending messages (sometimes as few as one message but most of the time is is hundreds of messages) the system becomes unresponsive. The strange thing is that it is not rabbitmq-server that is at fault it is systemd-journal. For some reason (I have not idea why) systemd-journal has decided to do something that takes 100% CPU (it appears to be using the SDCARD although I am not certain of it) and frequently will also consume all of RAM including all of the swap space, this then leads to other applications crashing including kernel crashes. I have been unable to determine what systemd-journal is doing. We have never seen systemd do this before and until I started using RabbitMQ I have not had any problems with systemd-journal. The strangest part is that rabbitmq is working just fine, it is not complaining about anything and I can not find any errors or failures any where until the RAM runs out due to systemd-journal taking all of it. My best guest is that rabbitmq is dumping a very large volume of log messages that systemd-journal is having to deal with but I can not find anything to indicate that is what is happening since the journal does not show any log entries being added other than the normal low volume of entries. I am hoping that someone might have an idea of what is going on? and maybe even some idea about how to fix it. I am using the Arch distro current as of a few months ago.
shaman2014
 
Posts: 3
Joined: Tue Aug 26, 2014 4:54 am

Return to Arch Linux ARM

Who is online

Users browsing this forum: No registered users and 3 guests