[ceph-users] Best way to remove an OSD node

[Caspar Smit]

Hi All,

What would be the best way to remove an entire OSD node from a cluster?

I've ran into problems removing OSD's from that node 1 by 1, eventually the
last few OSD's are overloaded with data.

Setting the crush weight of all these OSD's to 0 at once seems a bit
rigorous
Is there also a gentle (balanced) way to slowly move data off that node?

Something like:

ceph osd crush reweight <all OSD's to remove> 0.8 (all at once?)
then to 60%, 40%, 20% and eventually 0

Kind regards,
Caspar

[John Petrini]

There's a gentle reweight python script floating around on the net that
does this. It gradually reduces the weight of each osd one by one waiting
for rebalance to complete each time.

I've never used it and it may not work on all versions so I'd make sure to
test it.

That or do it manually but that's a tedius process.

Post by Caspar Smit
Hi All,
What would be the best way to remove an entire OSD node from a cluster?
I've ran into problems removing OSD's from that node 1 by 1, eventually
the last few OSD's are overloaded with data.
Setting the crush weight of all these OSD's to 0 at once seems a bit
rigorous
Is there also a gentle (balanced) way to slowly move data off that node?
ceph osd crush reweight <all OSD's to remove> 0.8 (all at once?)
then to 60%, 40%, 20% and eventually 0
Kind regards,
Caspar
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Caspar Smit]

Hi John,

Thanks for pointing out that script, do you have a link to it? I'm not able
to find it.
Just want to look at the script to understand its strategy.

Kind regards,
Caspar Smit

Post by John Petrini
There's a gentle reweight python script floating around on the net that
does this. It gradually reduces the weight of each osd one by one waiting
for rebalance to complete each time.
I've never used it and it may not work on all versions so I'd make sure to
test it.
That or do it manually but that's a tedius process.

Post by Caspar Smit
Hi All,
What would be the best way to remove an entire OSD node from a cluster?
I've ran into problems removing OSD's from that node 1 by 1, eventually
the last few OSD's are overloaded with data.
Setting the crush weight of all these OSD's to 0 at once seems a bit
rigorous
Is there also a gentle (balanced) way to slowly move data off that node?
ceph osd crush reweight <all OSD's to remove> 0.8 (all at once?)
then to 60%, 40%, 20% and eventually 0
Kind regards,
Caspar
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Serkan Çoban]

Here it is:
https://github.com/cernceph/ceph-scripts/blob/master/tools/ceph-gentle-reweight

Post by Caspar Smit
Hi John,
Thanks for pointing out that script, do you have a link to it? I'm not able
to find it.
Just want to look at the script to understand its strategy.
Kind regards,
Caspar Smit

Post by John Petrini
There's a gentle reweight python script floating around on the net that
does this. It gradually reduces the weight of each osd one by one waiting
for rebalance to complete each time.
I've never used it and it may not work on all versions so I'd make sure to
test it.
That or do it manually but that's a tedius process.

Post by Caspar Smit
Hi All,
What would be the best way to remove an entire OSD node from a cluster?
I've ran into problems removing OSD's from that node 1 by 1, eventually
the last few OSD's are overloaded with data.
Setting the crush weight of all these OSD's to 0 at once seems a bit
rigorous
Is there also a gentle (balanced) way to slowly move data off that node?
ceph osd crush reweight <all OSD's to remove> 0.8 (all at once?)
then to 60%, 40%, 20% and eventually 0
Kind regards,
Caspar
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com