Discussion:
Changing osd crush chooseleaf type at runtime
Add Reply
Flemming Frandsen
2018-02-02 12:47:36 UTC
Reply
Permalink
Raw Message
Hi, I'm just starting to play around with Ceph, so please excuse my
complete lack of a clue if this question is covered somewhere, but I
have been unable to find an answer.


I have a single machine running Ceph which was set up with osd crush
chooseleaf type = 0 in /etc/ceph/ceph.conf, now I've added a new machine
with some new OSDs, so I'd like to change to osd crush chooseleaf type =
1 and have Ceph re-balance the replicas.

How do I do that?

Preferably I'd like to make the change without making the cluster
unavailable.


So far I've edited the config file and tried restarting daemons,
including rebooting the entire OS, but I still see PGs that live only on
one host.

I've read the config docuemntation page but it doesn't mention what to
do to make that specific config change take effect:

http://docs.ceph.com/docs/master/rados/configuration/ceph-conf/

I've barked up the crushmap tree a bit, but I did not see how "osd crush
chooseleaf type" relates to that in any way.
--
Regards Flemming Frandsen - Stibo Systems - DK - STEP Release Manager
Please use ***@stibo.com for all Release Management requests
Gregory Farnum
2018-02-02 16:07:22 UTC
Reply
Permalink
Raw Message
Once you've created a crush map you need to edit it directly (either by
dumping it from the cluster, editing with the crush tool, and importing; or
via the ceph cli commands), rather than by updating config settings. I
believe doing so is explained in the ceph docs.

On Fri, Feb 2, 2018 at 4:47 AM Flemming Frandsen <
Post by Flemming Frandsen
Hi, I'm just starting to play around with Ceph, so please excuse my
complete lack of a clue if this question is covered somewhere, but I
have been unable to find an answer.
I have a single machine running Ceph which was set up with osd crush
chooseleaf type = 0 in /etc/ceph/ceph.conf, now I've added a new machine
with some new OSDs, so I'd like to change to osd crush chooseleaf type =
1 and have Ceph re-balance the replicas.
How do I do that?
Preferably I'd like to make the change without making the cluster
unavailable.
So far I've edited the config file and tried restarting daemons,
including rebooting the entire OS, but I still see PGs that live only on
one host.
I've read the config docuemntation page but it doesn't mention what to
http://docs.ceph.com/docs/master/rados/configuration/ceph-conf/
I've barked up the crushmap tree a bit, but I did not see how "osd crush
chooseleaf type" relates to that in any way.
--
Regards Flemming Frandsen - Stibo Systems - DK - STEP Release Manager
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Flemming Frandsen
2018-02-06 11:28:52 UTC
Reply
Permalink
Raw Message
Ah! Right, I guess my actual question was:

How does osd crush chooseleaf type = 0 and 1 alter the crushmap?


By experimentation I've figured out that:

"osd crush chooseleaf type = 0" turns into "step choose firstn 0 type
osd" and

"osd crush chooseleaf type = 1" turns into "step chooseleaf firstn 0
type host".


Changing the crushmap in this way worked perfectly for me, ceph -s
complained while doing the rebalancing, but eventually became happy with
the result.
Post by Gregory Farnum
Once you've created a crush map you need to edit it directly (either
by dumping it from the cluster, editing with the crush tool, and
importing; or via the ceph cli commands), rather than by updating
config settings. I believe doing so is explained in the ceph docs.
On Fri, Feb 2, 2018 at 4:47 AM Flemming Frandsen
Hi, I'm just starting to play around with Ceph, so please excuse my
complete lack of a clue if this question is covered somewhere, but I
have been unable to find an answer.
I have a single machine running Ceph which was set up with osd crush
chooseleaf type = 0 in /etc/ceph/ceph.conf, now I've added a new machine
with some new OSDs, so I'd like to change to osd crush chooseleaf type =
1 and have Ceph re-balance the replicas.
How do I do that?
Preferably I'd like to make the change without making the cluster
unavailable.
So far I've edited the config file and tried restarting daemons,
including rebooting the entire OS, but I still see PGs that live only on
one host.
I've read the config docuemntation page but it doesn't mention what to
http://docs.ceph.com/docs/master/rados/configuration/ceph-conf/
I've barked up the crushmap tree a bit, but I did not see how "osd crush
chooseleaf type" relates to that in any way.
--
Regards Flemming Frandsen - Stibo Systems - DK - STEP Release Manager
Release Management requests
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
Regards Flemming Frandsen - Stibo Systems - DK - STEP Release Manager
Please use ***@stibo.com for all Release Management requests
Loading...