Discussion:
rbd feature overheads
(too old to reply)
Blair Bethwaite
2018-02-12 05:25:08 UTC
Permalink
Hi all,

Wondering if anyone can clarify whether there are any significant overheads
from rbd features like object-map, fast-diff, etc. I'm interested in both
performance overheads from a latency and space perspective, e.g., can
object-map be sanely deployed on a 100TB volume or does the client try to
read the whole thing into memory...?
--
Cheers,
~Blairo
Ilya Dryomov
2018-02-12 10:01:27 UTC
Permalink
On Mon, Feb 12, 2018 at 6:25 AM, Blair Bethwaite
Post by Blair Bethwaite
Hi all,
Wondering if anyone can clarify whether there are any significant overheads
from rbd features like object-map, fast-diff, etc. I'm interested in both
performance overheads from a latency and space perspective, e.g., can
object-map be sanely deployed on a 100TB volume or does the client try to
read the whole thing into memory...?
Yes, it does. Enabling object-map on images larger than 1PB isn't
allowed for exactly that reason. The memory overhead is 2 bits per
object, i.e. 64K per 1TB assuming the default object size.

object-map also depends on exclusive-lock, which is bad for use cases
where sharing the same image between multiple clients is a requirement.

Once object-map is enabled, fast-diff is virtually no overhead.

Thanks,

Ilya
Blair Bethwaite
2018-02-13 00:24:28 UTC
Permalink
Thanks Ilya,

We can probably handle ~6.2MB for a 100TB volume. Is it reasonable to
expect a librbd client such as QEMU to only hold one object-map per guest?

Cheers,
Post by Ilya Dryomov
On Mon, Feb 12, 2018 at 6:25 AM, Blair Bethwaite
Post by Blair Bethwaite
Hi all,
Wondering if anyone can clarify whether there are any significant
overheads
Post by Blair Bethwaite
from rbd features like object-map, fast-diff, etc. I'm interested in both
performance overheads from a latency and space perspective, e.g., can
object-map be sanely deployed on a 100TB volume or does the client try to
read the whole thing into memory...?
Yes, it does. Enabling object-map on images larger than 1PB isn't
allowed for exactly that reason. The memory overhead is 2 bits per
object, i.e. 64K per 1TB assuming the default object size.
object-map also depends on exclusive-lock, which is bad for use cases
where sharing the same image between multiple clients is a requirement.
Once object-map is enabled, fast-diff is virtually no overhead.
Thanks,
Ilya
--
Cheers,
~Blairo
Ilya Dryomov
2018-02-13 09:41:00 UTC
Permalink
On Tue, Feb 13, 2018 at 1:24 AM, Blair Bethwaite
Post by Blair Bethwaite
Thanks Ilya,
We can probably handle ~6.2MB for a 100TB volume. Is it reasonable to expect
a librbd client such as QEMU to only hold one object-map per guest?
Yes, I think so.

Thanks,

Ilya

Loading...