概述
OSS: Object Storage Service
RadosGW兼容S3,我们需要依S3的方式提供OSS
OSS功能列表
Bucket相关
Buckets功能 | REST API | Ceph Operation & Class |
---|---|---|
List Buckets | GET / HTTP/1.1 Host: cname.company.cn Authorization: AWS {access-key}:{hash-of-header-and-secret} |
RGW_OP_LIST_BUCKETS class RGWListBuckets_ObjStore_S3 |
Put Bucket | PUT /{bucket} HTTP/1.1 Host: cname.company.cn x-amz-acl: public-read-write Authorization: AWS {access-key}:{hash-of-header-and-secret} |
RGW_OP_CREATE_BUCKET class RGWCreateBucket_ObjStore_S3 |
Delete Bucket | DELETE /{bucket} HTTP/1.1 Host: cname.company.cn Authorization: AWS {access-key}:{hash-of-header-and-secret} |
RGW_OP_DELETE_BUCKET class RGWDeleteBucket_ObjStore_S3 |
List Bucket Objects | GET /{bucket}?max-keys=25 HTTP/1.1 Host: cname.company.cn |
RGW_OP_LIST_BUCKET class RGWListBucket_ObjStore_S3 |
Get Bucket Location | GET /{bucket}?location HTTP/1.1 Host: cname.company.cn Authorization: AWS {access-key}:{hash-of-header-and-secret} |
class RGWGetBucketLocation_ObjStore_S3 |
Get Bucket ACL | GET /{bucket}?acl HTTP/1.1 Host: cname.company.cn Authorization: AWS {access-key}:{hash-of-header-and-secret} |
RGW_OP_GET_ACLS class RGWGetACLs_ObjStore_S3 |
Put Bucket ACL | PUT /{bucket}?acl HTTP/1.1 | RGW_OP_PUT_ACLS class RGWPutACLs_ObjStore_S3 |
List Bucket MultiPart Uploads | GET /{bucket}?uploads HTTP/1.1 | RGW_OP_LIST_BUCKET_MULTIPARTS class RGWListBucketMultiparts_ObjStore_S3 |
Head Bucket | HEAD / HTTP/1.1 Host: cname.company.cn Authorization: AWS {access-key}:{hash-of-header-and-secret} |
RGW_OP_STAT_BUCKET class RGWStatBucket_ObjStore_S3 |
Object相关
Object功能 | RESTful API | Ceph Operation & Class |
---|---|---|
Put Object | PUT /{bucket}/{object} HTTP/1.1 | RGW_OP_PUT_OBJ class RGWPutObj_ObjStore_S3 |
Copy Object | PUT /{dest-bucket}/{dest-object} HTTP/1.1 x-amz-copy-source: {source-bucket}/{source-object} |
RGW_OP_COPY_OBJ class RGWCopyObj_ObjStore_S3 |
Remove Object | DELETE /{bucket}/{object} HTTP/1.1 | RGW_OP_DELETE_OBJ class RGWDeleteObj_ObjStore_S3 |
Get Object | GET /{bucket}/{object} HTTP/1.1 | RGW_OP_GET_OBJ class RGWGetObj_ObjStore_S3 |
Get Object Info | HEAD /{bucket}/{object} HTTP/1.1 | RGW_OP_GET_OBJ class RGWGetObj_ObjStore_S3 |
Get Object ACL | GET /{bucket}/{object}?acl HTTP/1.1 | RGW_OP_GET_ACLS class RGWGetACLs_ObjStore_S3 |
Set Object ACL | PUT /{bucket}/{object}?acl | RGW_OP_PUT_ACLS class RGWPutACLs_ObjStore_S3 |
Initiate MultiPart Upload | POST /{bucket}/{object}?uploads | RGW_OP_INIT_MULTIPART class RGWInitMultipart_ObjStore_S3 |
MultiPart Upload Part | PUT /{bucket}/{object}?partNumber=&uploadId= HTTP/1.1 | RGW_OP_PUT_OBJ class RGWPutObj_ObjStore_S3 |
List MultiPart Upload Parts | GET /{bucket}/{object}?uploadId=123 HTTP/1.1 | RGW_OP_LIST_MULTIPART class RGWListMultipart_ObjStore_S3 |
Complete MultiPart Upload | POST /{bucket}/{object}?uploadId= HTTP/1.1 | RGW_OP_COMPLETE_MULTIPART class RGWCompleteMultipart_ObjStore_S3 |
Abort MultiPart Upload | DELETE /{bucket}/{object}?uploadId= HTTP/1.1 | RGW_OP_ABORT_MULTIPART class RGWAbortMultipart_ObjStore_S3 |
如何访问对象存储?
REST API
Service:
- GET
Bucket:
- GET
- <null>
- logging
- location
- versioning
- acl
- cors
- uploads
- PUT
- <null>
- logging
- versioning
- acl
- cors
- DELETE
- <null>
- cors
- HEAD
- <null>
- acl
- uploads
- POST
- <null>
- delete
- OPTIONS
- <null>
Object:
- GET
- <null>
- acl
- uploadId
- PUT
- acl
- copy_source
- DELETE
- <null>
- uploadId
- HEAD
- <null>
- acl
- uploadId
- POST
- uploadId
- uploads
- OPTIONS
- <null>
CLI命令行工具
s3cmd工具:
apt-get install s3cmd
s3cmd --configure
s3cmd --help
SDK包
兼容AWS S3提供的SDK包,但有部分功能不支持。
GUI管理界面
需要前端支持添加GUI管理界面。
RadosGW的用户帐号
user类型
There are two user types:
User: The term ‘user’ reflects a user of the S3 interface.
Subuser: The term ‘subuser’ reflects a user of the Swift interface. A subuser is associated to a user .
user操作
CREATE A USER
1
adosgw-admin user create --uid={username} --display-name="{display-name}" \[--email={email}\]
GET USER INFO
1
radosgw-admin user info --uid=johndoe
MODIFY USER INFO
1
radosgw-admin user modify --uid=johndoe --display-name="John E. Doe"
USER ENABLE/SUSPEND
1
2radosgw-admin user suspend --uid=johndoe
radosgw-admin user enable --uid=johndoeREMOVE A USER
1
radosgw-admin user rm --uid=johndoe
RadosGW兼容Keystone认证
官网上指出RadosGW兼容Openstack KeyStone认证,
http://docs.ceph.com/docs/hammer/radosgw/keystone/
但搜索发现Mirantis分析测试了RGW with Keystone,并不推荐这么做。
RGW中S3的认证方式
Keystone-based (disable default)
如何配置:
1
2[client.radosgw.gateway]
rgw s3 auth use keystone = trueRADOS-based(internal)
S3使用KeyStone认证的优缺点
优点
所有认证存储在统一的Keystone
不需要配置额外的S3认证管理系统,可以用Horizon替代
缺点
需要提升Keystone的性能以支持S3的请求
因为Keystone认证方式优先于内部的RADOS认证,则打开Keystone认证会使所有的S3认证先走KeyStone认证方式,如果失败了再尝试RADOS认证。这样使得正常使用S3
RADOS认证的请求时延增大,影响S3的性能。S3频繁访问Keystone服务,可能影响其他的Openstack service
我们如何使用?
个人推荐不使用Keystone认证S3的方式,而使用RadosGW内部的认证机制比较好。
但这样就引入了我们的注册用户如何使用S3的问题,结合阿里云,金山云,可以做如下实现:
类似阿里云,用户默认不能使用S3功能,需要点击“开通对象存储”按钮。
可以在开通对象存储过程中,给用户创建对应的S3 user和AccessKey/SecretKey对,与前端帐号信息绑定。
金山云用户注册后,登录对象存储跳转到单独的控制台界面,里面的帐号设置里就有两个AccessKey/SecretKey对。
这种方式也需要AccessKey/SecretKey对与前端帐号信息绑定。
参考资料:
https://content.mirantis.com/rs/451-RBY-185/images/Mirantis-Technical-Bulletin-S3-API-Keystone-integration-in-Ceph-RADOS-Gateway.pdf
http://dolphm.com/benchmarking-openstack-keystone-token-formats/
RadosGW提供OSS服务功能
基于上面的分析,使用RadosGW我们可以提供如下OSS功能,
对象存储OSSV1.0.0版本功能具体包括哪些?还需找再讨论确定。
分类 | 描述 | 操作 | Amazon | 金山云 | 我们公司 |
---|---|---|---|---|---|
Service Operation | 获取所有bucket信息 | GET Service | √ | √ | √ |
Bucket Operation | Bucket基本操作 | DELETE Bucket | √ | √ | √ |
GET Bucket | √ | √ | √ | ||
HEAD Bucket | √ | √ | √ | ||
PUT Bucket | √ | √ | √ | ||
Bucket cors相关操作 | DELETE Bucket cors | √ | √ | √ | |
GET Bucket cors | √ | √ | √ | ||
PUT Bucket cors | √ | √ | √ | ||
Bucket lifecycle相关操作 | DELETE Bucket lifecycle | √ | × | × | |
GET Bucket lifecycle | √ | ||||
PUT Bucket lifecycle | √ | ||||
Bucket policy相关操作 | DELETE Bucket policy | √ | × | × | |
GET Bucket policy | √ | ||||
PUT Bucket policy | √ | ||||
Bucket tagging相关操作 | DELETE Bucket tagging | √ | × | × | |
GET Bucket tagging | √ | ||||
PUT Bucket tagging | √ | ||||
Bucket website相关操作 | DELETE Bucket website | √ | × | hammer: × | |
jewel: √ | |||||
GET Bucket website | √ | ||||
PUT Bucket website | √ | ||||
Bucket logging相关操作 | GET Bucket logging | √ | √ | √ | |
PUT Bucket logging | √ | √ | √ | ||
Bucket notification相关操作 | GET Bucket notification | √ | × | × | |
PUT Bucket notification | √ | ||||
Bucket versioning相关操作 | GET Bucket versioning | √ | × | √ | |
GET Bucket Object versions | √ | hammer: × | |||
jewel: √ | |||||
PUT Bucket versioning | √ | √ | |||
Bucket acl相关操作 | PUT Bucket acl | √ | √ | √ | |
GET Bucket acl | √ | √ | √ | ||
Bucket requestPayment相关操作 | GET Bucket requestPayment | √ | × | hammer: × | |
jewel: √ | |||||
PUT Bucket requestPayment | √ | ||||
枚举该Bucket下的所有分块上传 | List MultiPart Uploads | √ | √ | √ | |
Object Operation | 删除Object | DELETE Object | √ | √ | √ |
删除多个Object | Delete Multiple Objects | √ | √ | √ | |
下载Object | GET Object | √ | √ | √ | |
获取Object ACL | GET Object ACL | √ | √ | √ | |
获取Object BT 种子 | GET Object torrent | √ | × | × | |
获取Object 元信息 | HEAD Object | √ | √ | √ | |
Object对HTML5浏览器跨域支持 | OPTIONS Object | √ | × | × | |
浏览器表单上传Object | POST Object | √ | √ | ×? | |
Amazon Glacier存储恢复 | POST Object restore | √ | × | × | |
上传Object | PUT Object | √ | √ | √ | |
设置Object ACL | PUT Object acl | √ | √ | √ | |
复制Object | PUT Object - Copy | √ | √ | √ | |
分块上传相关操作 | Initiate Multipart Upload | √ | √ | √ | |
Upload Part | √ | √ | √ | ||
Upload Part - Copy | √ | × | × | ||
Complete Multipart Upload | √ | √ | √ | ||
Abort Multipart Upload | √ | √ | √ | ||
List Parts | √ | √ | √ | ||
Image Thumbnail | × | √ | × |