[ceph-users] mount rbd read only

Discussion:

ST Wong (ITSC)

2018-11-08 12:05:33 UTC

Hi,

We created a testing rbd block device image as following:

----- cut here -------
# rbd create 4copy/foo --size 10G
# rbd feature disable 4copy/foo object-map fast-diff deep-flatten
# rbd --image 4copy/foo info
rbd image 'foo':
size 10 GiB in 2560 objects
order 22 (4 MiB objects)
id: 122f36b8b4567
block_name_prefix: rbd_data.122f36b8b4567
format: 2
features: layering, exclusive-lock
op_features:
flags:
create_timestamp: Thu Nov 8 19:42:25 2018

----- cut here -------

Then try to mount it on client but got error and can't be mounted:

----- cut here -------
# mount /dev/rbd0 /mnt
mount: /dev/rbd0 is write-protected, mounting read-only
mount: unknown filesystem type '(null)'
----- cut here -------

Did we do any step incorrect? We're using mimic. Thanks.

Besides, the rbd client is deployed through ceph-ansible as client role and found that the ceph.client.admin.keyring from admin server was also copied to the client machine. Is it necessary? Thanks a lot.

Best Regards,
/ST Wong

Ashley Merrick

2018-11-08 12:08:13 UTC

Permalink

What command are you using to mount the /dev/rbd0 to start with? You seem
to have missed that on your copy and paste.

Post by ST Wong (ITSC)
Hi,
----- cut here -------
# rbd create 4copy/foo --size 10G
# rbd feature disable 4copy/foo object-map fast-diff deep-flatten
# rbd --image 4copy/foo info
size 10 GiB in 2560 objects
order 22 (4 MiB objects)
id: 122f36b8b4567
block_name_prefix: rbd_data.122f36b8b4567
format: 2
features: layering, exclusive-lock
create_timestamp: Thu Nov 8 19:42:25 2018
----- cut here -------
----- cut here -------
# mount /dev/rbd0 /mnt
mount: /dev/rbd0 is write-protected, mounting read-only
mount: unknown filesystem type '(null)'
----- cut here -------
Did we do any step incorrect? Weâre using mimic. Thanks.
Besides, the rbd client is deployed through ceph-ansible as client role
and found that the ceph.client.admin.keyring from admin server was also
copied to the client machine. Is it necessary? Thanks a lot.
Best Regards,
/ST Wong
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Wido den Hollander

2018-11-08 12:30:41 UTC

Permalink

Did you create a filesystem on it with mkfs? Are you sure there is a
FileSystem on it?

Wido

Post by ST Wong (ITSC)
----- cut here -------

Did we do any step incorrect? We’re using mimic. Thanks.

Besides, the rbd client is deployed through ceph-ansible as client role
and found that the ceph.client.admin.keyring from admin server was also
copied to the client machine. Is it necessary? Thanks a lot.

Best Regards,
/ST Wong
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

ST Wong (ITSC)

2018-11-09 14:47:10 UTC

Permalink

Stupid me. I was focus on learning CEPH commands and forget something basic - haven't done mkfs. Sorry for the trouble caused.

Btw, is ceph.client.admin.keyring a must on client that mount rbd device? Any security concern?

Sorry for the newbie questions.
Thanks for all responded.

Best Rgds
/st wong

-----Original Message-----
From: ceph-users <ceph-users-***@lists.ceph.com> On Behalf Of Wido den Hollander
Sent: Thursday, November 8, 2018 8:31 PM
To: ceph-***@lists.ceph.com
Subject: Re: [ceph-users] mount rbd read only

Did you create a filesystem on it with mkfs? Are you sure there is a FileSystem on it?

Wido

Post by ST Wong (ITSC)
----- cut here -------

Did we do any step incorrect? We're using mimic. Thanks.

Besides, the rbd client is deployed through ceph-ansible as client
role and found that the ceph.client.admin.keyring from admin server
was also copied to the client machine. Is it necessary? Thanks a lot.

Best Regards,
/ST Wong
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Ashley Merrick

2018-11-09 14:51:04 UTC

Permalink

You could create a key ring that only has perms to mount the RBD and read
only to the monâs.

Depends if anyone that you wouldnât trust with ceph commands has access to
that VM / host.

Post by ST Wong (ITSC)
Stupid me. I was focus on learning CEPH commands and forget something
basic - haven't done mkfs. Sorry for the trouble caused.
Btw, is ceph.client.admin.keyring a must on client that mount rbd device?
Any security concern?
Sorry for the newbie questions.
Thanks for all responded.
Best Rgds
/st wong
-----Original Message-----
Sent: Thursday, November 8, 2018 8:31 PM
Subject: Re: [ceph-users] mount rbd read only

Did you create a filesystem on it with mkfs? Are you sure there is a FileSystem on it?
Wido

Post by ST Wong (ITSC)
----- cut here -------
Did we do any step incorrect? We're using mimic. Thanks.
Besides, the rbd client is deployed through ceph-ansible as client
role and found that the ceph.client.admin.keyring from admin server
was also copied to the client machine. Is it necessary? Thanks a lot.
Best Regards,
/ST Wong
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

ST Wong (ITSC)

2018-11-09 15:41:10 UTC

Permalink

Thanks for your help. Tried to follow steps in CEPH doc:

On admin host:

# ceph auth add client.acapp1 mon 'allow r' osd 'allow rw pool=4copy'
# ceph auth export client.acapp1 > keyring

Copy keyring to rbd client:/etc/ceph/keyring, and got following error:

# rbd map 4copy/foo
rbd: sysfs write failed
rbd: couldn't connect to the cluster!
In some cases useful info is found in syslog - try "dmesg | tail".
rbd: map failed: (22) Invalid argument

Also modified the capability as described in doc but gets same error:

# ceph auth caps client.acapp1 mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow pool templates r class-read, allow pool 4copy rwx'

Would you help? Thanks a lot.

Btw, shal /etc/ceph/ceph.client.admin.keyring be removed in ceph-ansible client deployment task?

Thanks and Best Regards,
/st wong

From: Ashley Merrick <***@amerrick.co.uk>
Sent: Friday, November 9, 2018 10:51 PM
To: ST Wong (ITSC) <***@itsc.cuhk.edu.hk>
Cc: Wido den Hollander <***@42on.com>; ceph-***@lists.ceph.com
Subject: Re: [ceph-users] mount rbd read only

You could create a key ring that only has perms to mount the RBD and read only to the monâs.

Depends if anyone that you wouldnât trust with ceph commands has access to that VM / host.

On Fri, 9 Nov 2018 at 10:47 PM, ST Wong (ITSC) <***@itsc.cuhk.edu.hk<mailto:***@itsc.cuhk.edu.hk>> wrote:
Stupid me. I was focus on learning CEPH commands and forget something basic - haven't done mkfs. Sorry for the trouble caused.

Btw, is ceph.client.admin.keyring a must on client that mount rbd device? Any security concern?

Sorry for the newbie questions.
Thanks for all responded.

Best Rgds
/st wong

-----Original Message-----
From: ceph-users <ceph-users-***@lists.ceph.com<mailto:ceph-users-***@lists.ceph.com>> On Behalf Of Wido den Hollander
Sent: Thursday, November 8, 2018 8:31 PM
To: ceph-***@lists.ceph.com<mailto:ceph-***@lists.ceph.com>
Subject: Re: [ceph-users] mount rbd read only

Did you create a filesystem on it with mkfs? Are you sure there is a FileSystem on it?

Wido

Post by ST Wong (ITSC)
----- cut here -------
Did we do any step incorrect? We're using mimic. Thanks.
Besides, the rbd client is deployed through ceph-ansible as client
role and found that the ceph.client.admin.keyring from admin server
was also copied to the client machine. Is it necessary? Thanks a lot.
Best Regards,
/ST Wong
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

_______________________________________________
ceph-users mailing list
ceph-***@lists.ceph.com<mailto:ceph-***@lists.ceph.com>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-***@lists.ceph.com<mailto:ceph-***@lists.ceph.com>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Ashley Merrick

2018-11-09 15:44:15 UTC

Permalink

You need to tell it the username and the key ring to use.

Iâm on my mobile right now so donât have access to a server to check but If
you check the man of the RBD command it is something like id/name.

If your key ring is named the correct format it will find the key ring, if
not you can specify the location using âkeyring

Post by ST Wong (ITSC)
# ceph auth add client.acapp1 mon 'allow r' osd 'allow rw pool=4copy'
# ceph auth export client.acapp1 > keyring
# rbd map 4copy/foo
rbd: sysfs write failed
rbd: couldn't connect to the cluster!
In some cases useful info is found in syslog - try "dmesg | tail".
rbd: map failed: (22) Invalid argument
# ceph auth caps client.acapp1 mon 'allow r' osd 'allow class-read
object_prefix rbd_children, allow pool templates r class-read, allow pool
4copy rwx'
Would you help? Thanks a lot.
Btw, shal /etc/ceph/ceph.client.admin.keyring be removed in ceph-ansible
client deployment task?
Thanks and Best Regards,
/st wong
*Sent:* Friday, November 9, 2018 10:51 PM
*Subject:* Re: [ceph-users] mount rbd read only
You could create a key ring that only has perms to mount the RBD and read
only to the monâs.
Depends if anyone that you wouldnât trust with ceph commands has access to
that VM / host.
Stupid me. I was focus on learning CEPH commands and forget something
basic - haven't done mkfs. Sorry for the trouble caused.
Btw, is ceph.client.admin.keyring a must on client that mount rbd device?
Any security concern?
Sorry for the newbie questions.
Thanks for all responded.
Best Rgds
/st wong
-----Original Message-----
Sent: Thursday, November 8, 2018 8:31 PM
Subject: Re: [ceph-users] mount rbd read only

Did you create a filesystem on it with mkfs? Are you sure there is a FileSystem on it?
Wido

Post by ST Wong (ITSC)
----- cut here -------
Did we do any step incorrect? We're using mimic. Thanks.
Besides, the rbd client is deployed through ceph-ansible as client
role and found that the ceph.client.admin.keyring from admin server
was also copied to the client machine. Is it necessary? Thanks a lot.
Best Regards,
/ST Wong
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

ST Wong (ITSC)

2018-11-12 03:40:48 UTC

Permalink

Thanks a lot.

Finally I can map it with following caps and map using non-default username :

-------
caps: [mgr] allow r
caps: [mon] allow r
caps: [osd] allow rw pool=4copy

# rbd -n client.acapp1 map 4copy/foo
/dev/rbd0
--------

Thanks a lot for all your help.

Btw, shall /etc/ceph/ceph.client.admin.keyring be removed in ceph-ansible client deployment task? Thanks.
Best Regards,
/st wong

From: Ashley Merrick <***@amerrick.co.uk>
Sent: Friday, November 9, 2018 11:44 PM
To: ST Wong (ITSC) <***@itsc.cuhk.edu.hk>
Cc: Wido den Hollander <***@42on.com>; ceph-***@lists.ceph.com
Subject: Re: [ceph-users] mount rbd read only

You need to tell it the username and the key ring to use.

Iâm on my mobile right now so donât have access to a server to check but If you check the man of the RBD command it is something like id/name.

If your key ring is named the correct format it will find the key ring, if not you can specify the location using âkeyring

On Fri, 9 Nov 2018 at 11:41 PM, ST Wong (ITSC) <***@itsc.cuhk.edu.hk<mailto:***@itsc.cuhk.edu.hk>> wrote:
Thanks for your help. Tried to follow steps in CEPH doc:

On admin host:

# ceph auth add client.acapp1 mon 'allow r' osd 'allow rw pool=4copy'
# ceph auth export client.acapp1 > keyring

Copy keyring to rbd client:/etc/ceph/keyring, and got following error:

# rbd map 4copy/foo
rbd: sysfs write failed
rbd: couldn't connect to the cluster!
In some cases useful info is found in syslog - try "dmesg | tail".
rbd: map failed: (22) Invalid argument

Also modified the capability as described in doc but gets same error:

# ceph auth caps client.acapp1 mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow pool templates r class-read, allow pool 4copy rwx'

Would you help? Thanks a lot.

Btw, shal /etc/ceph/ceph.client.admin.keyring be removed in ceph-ansible client deployment task?

Thanks and Best Regards,
/st wong

From: Ashley Merrick <***@amerrick.co.uk<mailto:***@amerrick.co.uk>>
Sent: Friday, November 9, 2018 10:51 PM
To: ST Wong (ITSC) <***@itsc.cuhk.edu.hk<mailto:***@itsc.cuhk.edu.hk>>
Cc: Wido den Hollander <***@42on.com<mailto:***@42on.com>>; ceph-***@lists.ceph.com<mailto:ceph-***@lists.ceph.com>

Subject: Re: [ceph-users] mount rbd read only

You could create a key ring that only has perms to mount the RBD and read only to the monâs.

Depends if anyone that you wouldnât trust with ceph commands has access to that VM / host.

On Fri, 9 Nov 2018 at 10:47 PM, ST Wong (ITSC) <***@itsc.cuhk.edu.hk<mailto:***@itsc.cuhk.edu.hk>> wrote:
Stupid me. I was focus on learning CEPH commands and forget something basic - haven't done mkfs. Sorry for the trouble caused.

Btw, is ceph.client.admin.keyring a must on client that mount rbd device? Any security concern?

Sorry for the newbie questions.
Thanks for all responded.

Best Rgds
/st wong

-----Original Message-----
From: ceph-users <ceph-users-***@lists.ceph.com<mailto:ceph-users-***@lists.ceph.com>> On Behalf Of Wido den Hollander
Sent: Thursday, November 8, 2018 8:31 PM
To: ceph-***@lists.ceph.com<mailto:ceph-***@lists.ceph.com>
Subject: Re: [ceph-users] mount rbd read only

Did you create a filesystem on it with mkfs? Are you sure there is a FileSystem on it?

Wido

Post by ST Wong (ITSC)
----- cut here -------
Did we do any step incorrect? We're using mimic. Thanks.
Besides, the rbd client is deployed through ceph-ansible as client
role and found that the ceph.client.admin.keyring from admin server
was also copied to the client machine. Is it necessary? Thanks a lot.
Best Regards,
/ST Wong
_______________________________________________
ceph-users mailing list
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com