[ceph-users] Large LOG like files on monitor

[Erwin Lubbers]

Hi,

In the /var/lib/ceph/mon/ceph-l16-s01/store.db/ directory there are two very large files LOG and LOG.OLD (multiple GB's) and my diskspace is running low. Can I safely delete those files?

Regards,
Erwin

[Christian Balzer]

Hello,

Post by Erwin Lubbers
Hi,
In the /var/lib/ceph/mon/ceph-l16-s01/store.db/ directory there are two
very large files LOG and LOG.OLD (multiple GB's) and my diskspace is
running low. Can I safely delete those files?

That sounds odd, what version of Ceph are you running, something
pre-Firefly?

How long has this monitor been running, what size, how busy is your
cluster?

Those leveldb log files shouldn't get that big and getting auto-rotated.

In general, if lsof shows that file being used (most likely if the date of
LOG is current) it obviously isn't safe to remove it.

Christian

Post by Erwin Lubbers
Regards,
Erwin
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Erwin Lubbers]

Christian,

Still running Dumpling (I know I have to start upgrading). Cluster has 66 OSD’s and a total size close to 100 GB. Cluster is running for around 2 years now and the monitor server has an uptime of 258 days.

The LOG file is 1.2 GB in size and ls shows the current time for it. The LOG.OLD is 1.5 GB and ls tells me a date of Jan 23 2015 (which is 258 before today).

lsof tells that the LOG is open, while the LOG.OLD isn’t.

So it don’t seems to rotate, without restarting the monitor.

Regards,
Erwin

Post by Christian Balzer
Hello,

Post by Erwin Lubbers
Hi,
In the /var/lib/ceph/mon/ceph-l16-s01/store.db/ directory there are two
very large files LOG and LOG.OLD (multiple GB's) and my diskspace is
running low. Can I safely delete those files?

That sounds odd, what version of Ceph are you running, something
pre-Firefly?
How long has this monitor been running, what size, how busy is your
cluster?
Those leveldb log files shouldn't get that big and getting auto-rotated.
In general, if lsof shows that file being used (most likely if the date of
LOG is current) it obviously isn't safe to remove it.
Christian

Post by Erwin Lubbers
Regards,
Erwin
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

--
Christian Balzer Network/Systems Engineer
http://www.gol.com/

[Christian Balzer]

Hello,

Post by Erwin Lubbers
Christian,
Still running Dumpling (I know I have to start upgrading). Cluster has
66 OSD’s and a total size close to 100 GB. Cluster is running for around
2 years now and the monitor server has an uptime of 258 days.
The LOG file is 1.2 GB in size and ls shows the current time for it. The
LOG.OLD is 1.5 GB and ls tells me a date of Jan 23 2015 (which is 258
before today).
lsof tells that the LOG is open, while the LOG.OLD isn’t.
So it don’t seems to rotate, without restarting the monitor.

Yup, that sounds about right.
Firefly will auto-rotate frequently, keeping logs in the few MB range.

Guess deleting the old one, restarting the monitor and then again deleting
the old, former log is your best way forward.

This is of course assuming that you have 3 or 5 MONs up and running.

Christian

Post by Erwin Lubbers
Regards,
Erwin

Post by Christian Balzer
Hello,

Post by Erwin Lubbers
Hi,
In the /var/lib/ceph/mon/ceph-l16-s01/store.db/ directory there are
two very large files LOG and LOG.OLD (multiple GB's) and my diskspace
is running low. Can I safely delete those files?

That sounds odd, what version of Ceph are you running, something
pre-Firefly?
How long has this monitor been running, what size, how busy is your
cluster?
Those leveldb log files shouldn't get that big and getting
auto-rotated.
In general, if lsof shows that file being used (most likely if the
date of LOG is current) it obviously isn't safe to remove it.
Christian

Post by Erwin Lubbers
Regards,
Erwin
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

--
Christian Balzer Network/Systems Engineer
http://www.gol.com/