[ceph-users] Disable intra-host replication?

[Marco Gaiarin]

Previous (partial) node failures and my current experiments on adding a
node lead me to the fact that, when rebalancing are needed, ceph
rebalance also on intra-node: eg, if an OSD of a node die, data are
rebalanced on all OSD, even if i've pool molteplicity 3 and 3 nodes.

This, indeed, make perfectly sense: overral data scattering have better
performance and safety.


But... there's some way to se to crush 'don't rebalance in the same node, go
in degradated mode'?


Thanks.

[Janne Johansson]

Post by Marco Gaiarin
Previous (partial) node failures and my current experiments on adding a
node lead me to the fact that, when rebalancing are needed, ceph
rebalance also on intra-node: eg, if an OSD of a node die, data are
rebalanced on all OSD, even if i've pool molteplicity 3 and 3 nodes.
This, indeed, make perfectly sense: overral data scattering have better
performance and safety.
But... there's some way to se to crush 'don't rebalance in the same node, go
in degradated mode'?

The default crush rules with replication=3 would only place PGs on
separate hosts,
so in that case it would go into degraded mode if a node goes away,
and not place
replicas on different disks on the remaining hosts.

[Marco Gaiarin]

Mandi! Janne Johansson
In chel di` si favelave...

Post by Janne Johansson
The default crush rules with replication=3 would only place PGs on
separate hosts,
so in that case it would go into degraded mode if a node goes away,
and not place
replicas on different disks on the remaining hosts.

'hosts' mean 'hosts with OSDs', right?

Because my cluster have 5 hosts, 2 are only MONs.


Thanks.

[Janne Johansson]

Post by Marco Gaiarin
Mandi! Janne Johansson
In chel di` si favelave...

Post by Janne Johansson
The default crush rules with replication=3 would only place PGs on
separate hosts,
so in that case it would go into degraded mode if a node goes away,
and not place
replicas on different disks on the remaining hosts.

'hosts' mean 'hosts with OSDs', right?
Because my cluster have 5 hosts, 2 are only MONs.

Yes, only hosts with OSDs.