Discussion:
radosgw not listening after installation
(too old to reply)
Piers Haken
2018-02-06 00:51:29 UTC
Permalink
i'm trying to setup radosgw on a brand new clutser, but I'm running into an issue where it's not listening on the default port (7480)

here's my install script:

ceph-deploy new $NODE
ceph-deploy install --release luminous $NODE
ceph-deploy install --release luminous --rgw $NODE
ceph-deploy mon create-initial
ceph-deploy admin $NODE
ceph-deploy rgw create $NODE

this is on debian 9.3 (stretch) on a clean machine.

the /usr/bin/radosgw process is running, and it's listening on port 6789 (this is not an HTTP server, but some insternal binary protocol), but the docs say it should be listening for HTTP requestst on port 7480.

what am i missing here?
Jean-Charles Lopez
2018-02-06 01:22:39 UTC
Permalink
Hi,

first of all just in case, it looks like your script does not deploy any OSDs as you go straight from MON to RGW.

then, RGW does listen by default on 7480 and what you see on 6789 is the MON listening.

Investigation:
- Make sure your ceph-radosgw process is running first.
- If not running, have a look at the log to see why it may have failed.
- Paste some more information in this mailing list so we can help you find the problem (e.g. output of ceph-deploy, log of your RGW, ...)

My bet is that given that you haven’t deployed any OSDs the RGW can’t create the pools it needs to stroe data. May be not but just guessing from what you showed us.

Regards
JC
Post by Piers Haken
i'm trying to setup radosgw on a brand new clutser, but I'm running into an issue where it's not listening on the default port (7480)
ceph-deploy new $NODE
ceph-deploy install --release luminous $NODE
ceph-deploy install --release luminous --rgw $NODE
ceph-deploy mon create-initial
ceph-deploy admin $NODE
ceph-deploy rgw create $NODE
this is on debian 9.3 (stretch) on a clean machine.
the /usr/bin/radosgw process is running, and it's listening on port 6789 (this is not an HTTP server, but some insternal binary protocol), but the docs say it should be listening for HTTP requestst on port 7480.
what am i missing here?
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com <http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com>
Piers Haken
2018-02-06 02:14:11 UTC
Permalink
Thanks, JC,

You’re right I didn’t deploy any OSDs at that point. I didn’t think that would be a problem since the last `ceph-deploy` command completed without error and its log ended with:

The Ceph Object Gateway (RGW) is now running on host storage-test01 and default port 7480

Maybe that’s a bug?


Anyway, I purged the cluster, rebuilt it with some OSDs, but I still don’t see radosgw listening on port 7480.

Here’s my ceph.conf (it’s just the default, I haven’t touched it):

[global]
fsid = 849f7b15-1e31-450b-b17c-6599fb6ff94d
mon_initial_members = storage-test01
mon_host = 10.0.4.127
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx

here’s my netstat:

# netstat -alp | grep rados

tcp 0 0 10.0.4.127:57504 10.0.4.127:6816 ESTABLISHED 19833/radosgw
tcp 0 0 10.0.4.127:43462 10.0.4.127:6832 ESTABLISHED 19833/radosgw
tcp 0 0 10.0.4.127:49848 10.0.4.127:6789 ESTABLISHED 19833/radosgw
unix 2 [ ACC ] STREAM LISTENING 262758 19833/radosgw /var/run/ceph/ceph-client.rgw.storage-test01.asok
unix 3 [ ] STREAM CONNECTED 264433 19833/radosgw

ps:

20243 ? Ssl 0:00 /usr/bin/radosgw -f --cluster ceph --name client.rgw.storage-test01 --setuser ceph --setgroup ceph


Sometimes there’s an intermittent “Initialization timeout, failed to initialize” in the rgw log, but it doesn’t occur when I restart the services.

Let me know if I can send anything else, I’d really like to get this up and running!

Thanks
Piers.


From: Jean-Charles Lopez [mailto:***@redhat.com]
Sent: Monday, February 05, 2018 5:23 PM
To: Piers Haken <***@hotmail.com>
Cc: ceph-***@lists.ceph.com
Subject: Re: [ceph-users] radosgw not listening after installation

Hi,

first of all just in case, it looks like your script does not deploy any OSDs as you go straight from MON to RGW.

then, RGW does listen by default on 7480 and what you see on 6789 is the MON listening.

Investigation:
- Make sure your ceph-radosgw process is running first.
- If not running, have a look at the log to see why it may have failed.
- Paste some more information in this mailing list so we can help you find the problem (e.g. output of ceph-deploy, log of your RGW, ...)

My bet is that given that you haven’t deployed any OSDs the RGW can’t create the pools it needs to stroe data. May be not but just guessing from what you showed us.

Regards
JC

On Feb 5, 2018, at 16:51, Piers Haken <***@hotmail.com<mailto:***@hotmail.com>> wrote:

i'm trying to setup radosgw on a brand new clutser, but I'm running into an issue where it's not listening on the default port (7480)

here's my install script:

ceph-deploy new $NODE
ceph-deploy install --release luminous $NODE
ceph-deploy install --release luminous --rgw $NODE
ceph-deploy mon create-initial
ceph-deploy admin $NODE
ceph-deploy rgw create $NODE

this is on debian 9.3 (stretch) on a clean machine.

the /usr/bin/radosgw process is running, and it's listening on port 6789 (this is not an HTTP server, but some insternal binary protocol), but the docs say it should be listening for HTTP requestst on port 7480.

what am i missing here?
_______________________________________________
ceph-users mailing list
ceph-***@lists.ceph.com<mailto:ceph-***@lists.ceph.com>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.ceph.com%2Flistinfo.cgi%2Fceph-users-ceph.com&data=02%7C01%7C%7Cc82c135cd4b04a359eef08d56d001f64%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636534769653549303&sdata=WfV19ft%2BydpuLRkWaNiuQv%2BiFO9An9FrH%2B47cIYe40Y%3D&reserved=0>
Jean-Charles Lopez
2018-02-06 03:00:56 UTC
Permalink
Hi

see inline

JC
Post by Piers Haken
Thanks, JC,
The Ceph Object Gateway (RGW) is now running on host storage-test01 and default port 7480
Maybe that’s a bug?
What version is this (ceph -v and ceph-deploy version)
Post by Piers Haken
Anyway, I purged the cluster, rebuilt it with some OSDs, but I still don’t see radosgw listening on port 7480.
[global]
fsid = 849f7b15-1e31-450b-b17c-6599fb6ff94d
mon_initial_members = storage-test01
mon_host = 10.0.4.127
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
May be some minimum pieces are missing from your config file assuming the outpout above is complete. It’s a long time I haven’t deployed with ceph-deploy
[client.rgw.storage-test01]
rgw_frontends = "civetweb port=7480" just to be sure
Post by Piers Haken
# netstat -alp | grep rados
tcp 0 0 10.0.4.127:57504 10.0.4.127:6816 ESTABLISHED 19833/radosgw
tcp 0 0 10.0.4.127:43462 10.0.4.127:6832 ESTABLISHED 19833/radosgw
tcp 0 0 10.0.4.127:49848 10.0.4.127:6789 ESTABLISHED 19833/radosgw
Here you can see that the RGW is connected to the MON (6789) not listening on 6789. The second column is the remote address the first one is the local address
Post by Piers Haken
unix 2 [ ACC ] STREAM LISTENING 262758 19833/radosgw /var/run/ceph/ceph-client.rgw.storage-test01.asok
unix 3 [ ] STREAM CONNECTED 264433 19833/radosgw
20243 ? Ssl 0:00 /usr/bin/radosgw -f --cluster ceph --name client.rgw.storage-test01 --setuser ceph --setgroup ceph
Sometimes there’s an intermittent “Initialization timeout, failed to initialize” in the rgw log, but it doesn’t occur when I restart the services.
I suspect that the RGW doesn’t fully initialize because it can’t create the necessary pools. Only way you can trace that is by bumping up the trace with debug_rgw = 20 in the configuration file and from there you should be able to see where it sits and what it did.
Post by Piers Haken
Let me know if I can send anything else, I’d really like to get this up and running!
Thanks
Piers.
Sent: Monday, February 05, 2018 5:23 PM
Subject: Re: [ceph-users] radosgw not listening after installation
Hi,
first of all just in case, it looks like your script does not deploy any OSDs as you go straight from MON to RGW.
then, RGW does listen by default on 7480 and what you see on 6789 is the MON listening.
- Make sure your ceph-radosgw process is running first.
- If not running, have a look at the log to see why it may have failed.
- Paste some more information in this mailing list so we can help you find the problem (e.g. output of ceph-deploy, log of your RGW, ...)
My bet is that given that you haven’t deployed any OSDs the RGW can’t create the pools it needs to stroe data. May be not but just guessing from what you showed us.
Regards
JC
i'm trying to setup radosgw on a brand new clutser, but I'm running into an issue where it's not listening on the default port (7480)
ceph-deploy new $NODE
ceph-deploy install --release luminous $NODE
ceph-deploy install --release luminous --rgw $NODE
ceph-deploy mon create-initial
ceph-deploy admin $NODE
ceph-deploy rgw create $NODE
this is on debian 9.3 (stretch) on a clean machine.
the /usr/bin/radosgw process is running, and it's listening on port 6789 (this is not an HTTP server, but some insternal binary protocol), but the docs say it should be listening for HTTP requestst on port 7480.
what am i missing here?
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.ceph.com%2Flistinfo.cgi%2Fceph-users-ceph.com&data=02%7C01%7C%7Cc82c135cd4b04a359eef08d56d001f64%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636534769653549303&sdata=WfV19ft%2BydpuLRkWaNiuQv%2BiFO9An9FrH%2B47cIYe40Y%3D&reserved=0>
Loading...