[ceph-users] Moving Ceph cluster to different network segment

Discussion:

Fred Yang

2014-06-11 19:08:04 UTC

We need to move Ceph cluster to different network segment for
interconnectivity between mon and osc, anybody has the procedure regarding
how that can be done? Note that the host name reference will be changed, so
originally the osd host referenced as cephnode1, in the new segment it will
be cephnode1-n.

Thanks,
Fred

Sent from my Samsung Galaxy S3
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140611/b194859e/attachment.htm>

John Wilkins

2014-06-12 17:21:49 UTC

Permalink

Fred,

I'm not sure it will completely answer your question, but I would
definitely have a look at:
http://ceph.com/docs/master/rados/operations/add-or-rm-mons/#changing-a-monitor-s-ip-address

There are some important steps in there for monitors.

Post by Fred Yang
We need to move Ceph cluster to different network segment for
interconnectivity between mon and osc, anybody has the procedure regarding
how that can be done? Note that the host name reference will be changed, so
originally the osd host referenced as cephnode1, in the new segment it will
be cephnode1-n.
Thanks,
Fred
Sent from my Samsung Galaxy S3
_______________________________________________
ceph-users mailing list
ceph-users at lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

--
John Wilkins
Senior Technical Writer
Intank
john.wilkins at inktank.com
(415) 425-9599
http://inktank.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140612/ac3e1e42/attachment.htm>

Fred Yang

2014-06-13 11:41:09 UTC

Permalink

Thanks, John.

That seems will take care of monitors, how about osd? Any idea how to
change IP addresses without triggering a resync?

Fred

Sent from my Samsung Galaxy S3

Post by John Wilkins
Fred,
I'm not sure it will completely answer your question, but I would
http://ceph.com/docs/master/rados/operations/add-or-rm-mons/#changing-a-monitor-s-ip-address
There are some important steps in there for monitors.

--
John Wilkins
Senior Technical Writer
Intank
john.wilkins at inktank.com
(415) 425-9599
http://inktank.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140613/5b39c44b/attachment.htm>

Wido den Hollander

2014-06-13 11:51:39 UTC

Permalink

Post by Fred Yang
Thanks, John.
That seems will take care of monitors, how about osd? Any idea how to
change IP addresses without triggering a resync?

IPs of OSDs are dynamic. Their IP is no part of the data distribution.
Simply renumber them and restart the daemon.

I suggest:

1. Stop OSD(s)
2. Renumber machine
3. Start OSD(s)

That should be all. There will be some recovery due to I/Os which
occurred between 1 and 3.

Wido

Post by Fred Yang
Fred
Sent from my Samsung Galaxy S3
On Jun 12, 2014 1:21 PM, "John Wilkins" <john.wilkins at inktank.com
Fred,
I'm not sure it will completely answer your question, but I would
http://ceph.com/docs/master/rados/operations/add-or-rm-mons/#changing-a-monitor-s-ip-address
There are some important steps in there for monitors.
On Wed, Jun 11, 2014 at 12:08 PM, Fred Yang <frederic.yang at gmail.com
We need to move Ceph cluster to different network segment for
interconnectivity between mon and osc, anybody has the procedure
regarding how that can be done? Note that the host name
reference will be changed, so originally the osd host referenced
as cephnode1, in the new segment it will be cephnode1-n.
Thanks,
Fred
Sent from my Samsung Galaxy S3
_______________________________________________
ceph-users mailing list
ceph-users at lists.ceph.com <mailto:ceph-users at lists.ceph.com>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
John Wilkins
Senior Technical Writer
Intank
john.wilkins at inktank.com <mailto:john.wilkins at inktank.com>
(415) 425-9599 <tel:%28415%29%20425-9599>
http://inktank.com
_______________________________________________
ceph-users mailing list
ceph-users at lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

--
Wido den Hollander
42on B.V.
Ceph trainer and consultant

Phone: +31 (0)20 700 9902
Skype: contact42on

Fred Yang

2014-06-13 12:10:08 UTC

Permalink

Wido,
So the cluster reference osd based on the hostname, or the GUID(hopefully)?
Note that I mentioned in original email the hostname associated to the IP
will also be changed as well, it will be as simple as changing IP and
restart osd? I remembered I tested in Dumpling a while ago and it didn't
work, this cluster is running on Emperor and not sure whether that will
make any difference.

Fred

Post by Wido den Hollander

Post by Fred Yang
Thanks, John.
That seems will take care of monitors, how about osd? Any idea how to
change IP addresses without triggering a resync?

IPs of OSDs are dynamic. Their IP is no part of the data distribution.
Simply renumber them and restart the daemon.
1. Stop OSD(s)
2. Renumber machine
3. Start OSD(s)
That should be all. There will be some recovery due to I/Os which occurred
between 1 and 3.
Wido
Fred

Post by Fred Yang
Sent from my Samsung Galaxy S3
On Jun 12, 2014 1:21 PM, "John Wilkins" <john.wilkins at inktank.com
Fred,
I'm not sure it will completely answer your question, but I would
http://ceph.com/docs/master/rados/operations/add-or-rm-
mons/#changing-a-monitor-s-ip-address
There are some important steps in there for monitors.
On Wed, Jun 11, 2014 at 12:08 PM, Fred Yang <frederic.yang at gmail.com
We need to move Ceph cluster to different network segment for
interconnectivity between mon and osc, anybody has the procedure
regarding how that can be done? Note that the host name
reference will be changed, so originally the osd host referenced
as cephnode1, in the new segment it will be cephnode1-n.
Thanks,
Fred
Sent from my Samsung Galaxy S3
_______________________________________________
ceph-users mailing list
ceph-users at lists.ceph.com <mailto:ceph-users at lists.ceph.com>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
John Wilkins
Senior Technical Writer
Intank
john.wilkins at inktank.com <mailto:john.wilkins at inktank.com>
(415) 425-9599 <tel:%28415%29%20425-9599>
http://inktank.com
_______________________________________________
ceph-users mailing list
ceph-users at lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

--
Wido den Hollander
42on B.V.
Ceph trainer and consultant
Phone: +31 (0)20 700 9902
Skype: contact42on
_______________________________________________
ceph-users mailing list
ceph-users at lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140613/9d753499/attachment.htm>

Jake Young

2014-06-13 13:17:26 UTC

Permalink

I recently changed IP and hostname of an osd node running dumpling and had
no problems.

You do need to have your ceph.conf file built correctly or your osds won't
start. Make sure the new IPs and new hostname are in there before you
change the IP.

The crushmap showed a new bucket (host name) containing the osds that were
moved and the original bucket remained in the crushmap, but with no
children. I was able to unlink the original bucket with no problem.

Jake

Post by Fred Yang
Wido,
So the cluster reference osd based on the hostname, or the
GUID(hopefully)? Note that I mentioned in original email the hostname
associated to the IP will also be changed as well, it will be as simple as
changing IP and restart osd? I remembered I tested in Dumpling a while ago
and it didn't work, this cluster is running on Emperor and not sure whether
that will make any difference.
Fred
On Jun 13, 2014 7:51 AM, "Wido den Hollander" <wido at 42on.com

Post by Wido den Hollander

Post by Fred Yang
Thanks, John.
That seems will take care of monitors, how about osd? Any idea how to
change IP addresses without triggering a resync?

Post by Fred Yang
Sent from my Samsung Galaxy S3
On Jun 12, 2014 1:21 PM, "John Wilkins" <john.wilkins at inktank.com
<javascript:_e(%7B%7D,'cvml','john.wilkins at inktank.com');>
<mailto:john.wilkins at inktank.com
Fred,
I'm not sure it will completely answer your question, but I would
http://ceph.com/docs/master/rados/operations/add-or-rm-
mons/#changing-a-monitor-s-ip-address
There are some important steps in there for monitors.
On Wed, Jun 11, 2014 at 12:08 PM, Fred Yang <frederic.yang at gmail.com
<javascript:_e(%7B%7D,'cvml','frederic.yang at gmail.com');>
<mailto:frederic.yang at gmail.com
We need to move Ceph cluster to different network segment for
interconnectivity between mon and osc, anybody has the procedure
regarding how that can be done? Note that the host name
reference will be changed, so originally the osd host referenced
as cephnode1, in the new segment it will be cephnode1-n.
Thanks,
Fred
Sent from my Samsung Galaxy S3
_______________________________________________
ceph-users mailing list
ceph-users at lists.ceph.com
ceph-users at lists.ceph.com
<javascript:_e(%7B%7D,'cvml','ceph-users at lists.ceph.com');>>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
John Wilkins
Senior Technical Writer
Intank
john.wilkins at inktank.com
john.wilkins at inktank.com
<javascript:_e(%7B%7D,'cvml','john.wilkins at inktank.com');>>
(415) 425-9599 <tel:%28415%29%20425-9599>
http://inktank.com
_______________________________________________
ceph-users mailing list
ceph-users at lists.ceph.com
<javascript:_e(%7B%7D,'cvml','ceph-users at lists.ceph.com');>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

--
Wido den Hollander
42on B.V.
Ceph trainer and consultant
Phone: +31 (0)20 700 9902
Skype: contact42on
_______________________________________________
ceph-users mailing list
ceph-users at lists.ceph.com
<javascript:_e(%7B%7D,'cvml','ceph-users at lists.ceph.com');>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140613/e287732d/attachment.htm>