Discussion:
Infinite loop in radosgw-usage show
Add Reply
Ingo Reimann
2018-02-02 06:04:00 UTC
Reply
Permalink
Raw Message
Hi,

after our jewel 10.2.10 -> luminous 12.2.2 upgrade of mons and osds but
not the rgws we got some nasty behavior.

Radosgw-admin usage show loops forever when there is any data. I can`t
reproduce it exactly, but for some users we got an output once during our
tests, but then it changed. When we surpress the output with
--show-log-entries=false, we don`t get any output at all, without that,
the first entries are repeated forever. I thought, this is
http://tracker.ceph.com/issues/21196, but this should have been closed in
12.2.12. We tried the command with jewel and luminous clients. Same
behaviour.

Any idea, how to get rid of that?


Best regards,

Ingo Reimann
Dunkel GmbH
Ingo Reimann
2018-02-06 08:04:22 UTC
Reply
Permalink
Raw Message
Just to add -

We wrote a little wrapper, that reads the output of "radosgw-admin usage
show" and stops, when the loop starts. When we add all entries by
ourselves, the result is correct. Moreover - the duplicate timestamp, that
we detect to break the loop, is not the last taken into account. Eg:
./radosgw-admin-break-loop --uid=TestUser --start-date=2017-12-01
--end-date=2018-01-01 [...]
"bytes_received": 1472051975516, Loop detected at
"2017-12-21 08:00:00.000000Z"

./radosgw-admin-break-loop --uid=TestUser --start-date=2017-12-01
--end-date=2017-12-22 [...]
"bytes_received": 1245051973424, Loop detected at
"2017-12-21 08:00:00.000000Z"

This leads to the assumption, that the loop occurs after processing of
raw data.

Looks like a bug?

Best regards,

Ingo Reimann
Dunkel GmbH
Orit Wasserman
2018-02-07 13:53:16 UTC
Reply
Permalink
Raw Message
Hi,
Post by Ingo Reimann
Just to add -
We wrote a little wrapper, that reads the output of "radosgw-admin usage
show" and stops, when the loop starts. When we add all entries by
ourselves, the result is correct. Moreover - the duplicate timestamp, that
./radosgw-admin-break-loop --uid=TestUser --start-date=2017-12-01
--end-date=2018-01-01 [...]
"bytes_received": 1472051975516, Loop detected at
"2017-12-21 08:00:00.000000Z"
./radosgw-admin-break-loop --uid=TestUser --start-date=2017-12-01
--end-date=2017-12-22 [...]
"bytes_received": 1245051973424, Loop detected at
"2017-12-21 08:00:00.000000Z"
This leads to the assumption, that the loop occurs after processing of
raw data.
Looks like a bug?
Yes :(
Please open a tracker issue please include your ceph version, special
configuration and logs.

Thanks,
Orit
Post by Ingo Reimann
Best regards,
Ingo Reimann
Dunkel GmbH
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Loading...