Upgrade Operator to 3.0.1 #201

dvaldivia · 2020-07-24T00:15:50Z

Fixes #199

bexsoft

Please fix Get Tenant Info:

--- FAIL: Test_TenantInfo (0.00s)
    --- FAIL: Test_TenantInfo/Get_tenant_Info (0.00s)

FAIL
FAIL	github.com/minio/mcs/restapi	13.231s
?   	github.com/minio/mcs/restapi/operations	[no test files]
?   	github.com/minio/mcs/restapi/operations/admin_api	[no test files]
?   	github.com/minio/mcs/restapi/operations/user_api	[no test files]
FAIL

go.mod

restapi/admin_tenants.go

restapi/admin_tenants_test.go

cesnietor · 2020-07-24T01:18:44Z

please also update the kustomize files if needed.

k8s/operator-console/base/minio-operator.yaml

dvaldivia · 2020-07-24T16:36:29Z

@bexsoft fixed

cesnietor · 2020-07-24T16:48:26Z

please make it a single commit

dvaldivia · 2020-07-24T16:56:18Z

@cesnietor done

cesnietor

please update the yaml permissions accordingly:

{
    "code": 500,
    "message": "tenants.minio.min.io is forbidden: User \"system:serviceaccount:default:mcs-sa\" cannot list resource \"tenants\" in API group \"minio.min.io\" at the cluster scope"
}

dvaldivia · 2020-07-24T17:41:22Z

@cesnietor fixed

bexsoft · 2020-07-24T17:51:54Z

Socket hangup trying to gather tenant information. (http://localhost:9090/api/v1/namespaces/default/tenants/minio-tenant-4)

goroutine 144 [running]:                                                                                                           │
│ net/http.(*conn).serve.func1(0xc0004fdea0)                                                                                         │
│     /usr/local/go/src/net/http/server.go:1767 +0x139                                                                               │
│ panic(0x1ae5fc0, 0x3180a00)                                                                                                        │
│     /usr/local/go/src/runtime/panic.go:679 +0x1b2                                                                                  │
│ github.com/minio/mcs/restapi.getTenantInfo(0xc000c94240, 0xc0007c1fe0, 0x213c880)                                                  │
│     /go/src/github.com/minio/mcs/restapi/admin_tenants.go:198 +0x3a3                                                               │
│ github.com/minio/mcs/restapi.getTenantInfoResponse(0xc0005c0870, 0xc000a8a800, 0xc000246217, 0x7, 0xc000246227, 0xe, 0x0, 0x0, 0x0 │
│     /go/src/github.com/minio/mcs/restapi/admin_tenants.go:259 +0x653                                                               │
│ github.com/minio/mcs/restapi.registerTenantHandlers.func4(0xc000a8a800, 0xc000246217, 0x7, 0xc000246227, 0xe, 0xc0005c0870, 0x1ca1 │
│     /go/src/github.com/minio/mcs/restapi/admin_tenants.go:77 +0x59                                                                 │
│ github.com/minio/mcs/restapi/operations/admin_api.TenantInfoHandlerFunc.Handle(0x1e6c110, 0xc000a8a800, 0xc000246217, 0x7, 0xc0002 │
│     /go/src/github.com/minio/mcs/restapi/operations/admin_api/tenant_info.go:38 +0x51                                              │
│ github.com/minio/mcs/restapi/operations/admin_api.(*TenantInfo).ServeHTTP(0xc000481960, 0x20ed5c0, 0xc0002762a0, 0xc000a8a800)     │
│     /go/src/github.com/minio/mcs/restapi/operations/admin_api/tenant_info.go:86 +0x342                                             │
│ github.com/go-openapi/runtime/middleware.NewOperationExecutor.func1(0x20ed5c0, 0xc0002762a0, 0xc000a8a700)                         │
│     /go/pkg/mod/github.com/go-openapi/[email protected]/middleware/operation.go:28 +0x75                                            │
│ net/http.HandlerFunc.ServeHTTP(0xc0006c17d0, 0x20ed5c0, 0xc0002762a0, 0xc000a8a700)                                                │
│     /usr/local/go/src/net/http/server.go:2007 +0x44                                                                                │
│ github.com/go-openapi/runtime/middleware.NewRouter.func1(0x20ed5c0, 0xc0002762a0, 0xc000a8a500)                                    │
│     /go/pkg/mod/github.com/go-openapi/[email protected]/middleware/router.go:77 +0x356                                              │
│ net/http.HandlerFunc.ServeHTTP(0xc000d157e0, 0x20ed5c0, 0xc0002762a0, 0xc000a8a500)                                                │
│     /usr/local/go/src/net/http/server.go:2007 +0x44                                                                                │
│ github.com/go-openapi/runtime/middleware.Redoc.func1(0x20ed5c0, 0xc0002762a0, 0xc000a8a500)                                        │
│     /go/pkg/mod/github.com/go-openapi/[email protected]/middleware/redoc.go:72 +0x286                                               │
│ net/http.HandlerFunc.ServeHTTP(0xc000cddb00, 0x20ed5c0, 0xc0002762a0, 0xc000a8a500)                                                │
│     /usr/local/go/src/net/http/server.go:2007 +0x44                                                                                │
│ github.com/go-openapi/runtime/middleware.Spec.func1(0x20ed5c0, 0xc0002762a0, 0xc000a8a500)                                         │
│     /go/pkg/mod/github.com/go-openapi/[email protected]/middleware/spec.go:46 +0x188                                                │
│ net/http.HandlerFunc.ServeHTTP(0xc000cddb40, 0x20ed5c0, 0xc0002762a0, 0xc000a8a500)                                                │
│     /usr/local/go/src/net/http/server.go:2007 +0x44                                                                                │
│ github.com/minio/mcs/restapi.FileServerMiddleware.func1(0x20ed5c0, 0xc0002762a0, 0xc000a8a500)                                     │
│     /go/src/github.com/minio/mcs/restapi/configure_mcs.go:184 +0x1e2                                                               │
│ net/http.HandlerFunc.ServeHTTP(0xc000d15c80, 0x20ed5c0, 0xc0002762a0, 0xc000a8a500)                                                │
│     /usr/local/go/src/net/http/server.go:2007 +0x44                                                                                │
│ github.com/unrolled/secure.(*Secure).Handler.func1(0x20ed5c0, 0xc0002762a0, 0xc000a8a500)                                          │
│     /go/pkg/mod/github.com/unrolled/[email protected]/secure.go:177 +0xdf                                                              │
│ net/http.HandlerFunc.ServeHTTP(0xc000d15ca0, 0x20ed5c0, 0xc0002762a0, 0xc000a8a500)                                                │
│     /usr/local/go/src/net/http/server.go:2007 +0x44                                                                                │
│ net/http.serverHandler.ServeHTTP(0xc000ce69a0, 0x20ed5c0, 0xc0002762a0, 0xc000a8a500)                                              │
│     /usr/local/go/src/net/http/server.go:2802 +0xa4                                                                                │
│ net/http.(*conn).serve(0xc0004fdea0, 0x20f4f00, 0xc0005d6140) ...

swagger.yml

cesnietor · 2020-07-24T17:53:45Z

Since A tenant can be created without serviceName, it now fails getting Tenant's info:
POST :

{
    "name": "minio-tenant-2",
    "namespace": "default",
    "zones": [
        {
            "servers": 1,
            "volumes_per_server": 1,
            "volume_configuration": {
                "storage_class_name": "standard",
                "size": "500Mi"
            }
        }
    ]
}

on GET:

{
    "code": 500,
    "message": "resource name may not be empty"
}

caused by :

error getting tenant's admin client: resource name may not be empty

cesnietor · 2020-07-24T17:59:08Z

Tenants are never created:

0-07-24T17:40:44.4224816Z I0724 17:40:44.411016       1 main-controller.go:320] Successfully synced 'default/minio-tenant-2'                                                                                                   │
│ 2020-07-24T17:40:44.4511057Z E0724 17:40:44.450764       1 main-controller.go:325] error syncing 'default/minio-tenant-2': Endpoint: :9000 does not follow ip address or domain name standards.      
 2020-07-24T17:37:29.0664031Z I0724 17:37:29.065859       1 main-controller.go:320] Successfully synced 'default/minio-tenant-1'                                                                                                   │
│ 2020-07-24T17:37:29.0881936Z E0724 17:37:29.087572       1 main-controller.go:325] error syncing 'default/minio-tenant-1': Endpoint: :9000 does not follow ip address or domain name standards.

cesnietor · 2020-07-24T18:08:14Z

Since this changes will affect UI it needs to also be addressed in this PR.

cesnietor · 2020-07-25T01:40:18Z

with the following request the minio pods are never starting:

{
    "name": "minio-tenant-2",
    "namespace": "default",
    "zones": [
        {
            "servers": 2,
            "volumes_per_server": 1,
            "volume_configuration": {
                "storage_class_name": "standard",
                "size": 1073741824
            }
        }
    ]
}

minio-tenant-2-zone-0-0           0/1     CrashLoopBackOff   7          13m
minio-tenant-2-zone-0-1           0/1     CrashLoopBackOff   7          13m

ERROR Invalid command line arguments: Incorrect number of endpoints provided [http://minio-tenant-2-zone-0-{0...1}.minio-tenant-2-hl.default.svc.cluster.local/export]
      > Please provide correct combination of local/remote paths
      HINT:
        For more information, please refer to https://docs.min.io/docs/minio-erasure-code-quickstart-guide

bexsoft

LGTM

harshavardhana

please upgrade to v3.0.1

k8s/operator-console/base/minio-operator.yaml

update all other dependent packages

cesnietor · 2020-07-25T21:08:27Z

If I create a Tenant, then I delete it (with its secrets) and then Create it again with the same request: the tenant doesn't start:
and the pods log:

│ 2020-07-25T21:05:18.7991578Z ERROR Unable to initialize server switching into safe-mode: Unable to initialize sub-systems: Unable to initialize config system: Invalid credentials                                                │
│

Request used:

{
    "name": "minio-tenant-3",
    "namespace": "default",
    "zones": [
        {
            "servers": 1,
            "volumes_per_server": 4,
            "volume_configuration": {
                "storage_class_name": "standard",
                "size": 1073741824
            }
        }
    ]
}

steps:
1-Create
2-DElete tenant
3- Delete secrets
4 - Repeat 1

harshavardhana · 2020-07-25T21:25:56Z

If I create a Tenant, then I delete it (with its secrets) and then Create it again with the same request: the tenant doesn't start:
and the pods log:

│ 2020-07-25T21:05:18.7991578Z ERROR Unable to initialize server switching into safe-mode: Unable to initialize sub-systems: Unable to initialize config system: Invalid credentials                                                │
│

Request used:

{
    "name": "minio-tenant-3",
    "namespace": "default",
    "zones": [
        {
            "servers": 1,
            "volumes_per_server": 4,
            "volume_configuration": {
                "storage_class_name": "standard",
                "size": 1073741824
            }
        }
    ]
}

steps:
1-Create
2-DElete tenant
3- Delete secrets
4 - Repeat 1

Secrets cannot be deleted, secrets are used for encrypting the config and IAM assets in MinIO so it is failing correctly. If you reuse same drives with different credentials.

Looks like an operator issue.

cesnietor · 2020-07-25T21:27:34Z

Secrets cannot be deleted

I meant the k8s secrets that mcs create

harshavardhana · 2020-07-25T21:28:59Z

Secrets cannot be deleted

I meant the k8s secrets that mcs create

If the same disks are used, same secrets should be used - I am explaining what is required for MinIO.

cesnietor · 2020-07-25T21:29:35Z

Looks like an operator issue.

I will approve this since we can update mcs with this minor bug and update once operator has the fix.

cesnietor

Tested, LGTM

harshavardhana · 2020-07-25T21:30:40Z

So it a tenant is deleted and somehow new secrets get added it is not allowed to start.

https://github.com/minio/minio/tree/master/docs/config#rotating-encryption-with-new-credentials

cesnietor · 2020-07-25T21:31:51Z

So it a tenant is deleted and somehow new secrets get added it is not allowed to start.

https://github.com/minio/minio/tree/master/docs/config#rotating-encryption-with-new-credentials

Oh I see.

dvaldivia requested review from Alevsk, cesnietor and bexsoft July 24, 2020 00:15

dvaldivia self-assigned this Jul 24, 2020

bexsoft requested changes Jul 24, 2020

View reviewed changes

cesnietor requested changes Jul 24, 2020

View reviewed changes

Alevsk previously approved these changes Jul 24, 2020

View reviewed changes

k8s/operator-console/base/minio-operator.yaml Show resolved Hide resolved

dvaldivia dismissed Alevsk’s stale review via 00836f8 July 24, 2020 16:52

dvaldivia force-pushed the upgade-operator branch from 4e999db to 00836f8 Compare July 24, 2020 16:52

Alevsk previously approved these changes Jul 24, 2020

View reviewed changes

cesnietor requested changes Jul 24, 2020

View reviewed changes

dvaldivia dismissed Alevsk’s stale review via 8db097b July 24, 2020 17:41

cesnietor requested changes Jul 24, 2020

View reviewed changes

swagger.yml Show resolved Hide resolved

dvaldivia force-pushed the upgade-operator branch 3 times, most recently from 51f7a1c to 31616b1 Compare July 24, 2020 23:29

dvaldivia mentioned this pull request Jul 24, 2020

Rename MCS to Console in codebase #205

Merged

Upgrade Operator to 3.0.0

0080751

dvaldivia force-pushed the upgade-operator branch from d6cdc7f to 0080751 Compare July 25, 2020 01:34

bexsoft previously approved these changes Jul 25, 2020

View reviewed changes

dvaldivia dismissed bexsoft’s stale review via e432183 July 25, 2020 18:20

Alevsk self-requested a review July 25, 2020 18:29

Alevsk previously approved these changes Jul 25, 2020

View reviewed changes

harshavardhana requested changes Jul 25, 2020

View reviewed changes

Update Operator References

5ecb311

dvaldivia dismissed Alevsk’s stale review via 5ecb311 July 25, 2020 18:51

harshavardhana requested changes Jul 25, 2020

View reviewed changes

k8s/operator-console/base/minio-operator.yaml Outdated Show resolved Hide resolved

dvaldivia and others added 8 commits July 25, 2020 12:01

Changes to operator v3

5dbdf35

Go mod with v3 tag for operator

16c25c0

Operator docker iamge

b631ab3

update all other dependent packages

001e2b5

Merge pull request #1 from minio/upgrade-operator

0cec745

update all other dependent packages

Add missing import

4475be0

Update Tests

afda190

lints

364175c

harshavardhana approved these changes Jul 25, 2020

View reviewed changes

harshavardhana requested a review from cesnietor July 25, 2020 19:57

Alevsk self-requested a review July 25, 2020 20:08

Alevsk approved these changes Jul 25, 2020

View reviewed changes

cesnietor approved these changes Jul 25, 2020

View reviewed changes

cesnietor changed the title ~~Upgrade Operator to 3.0.0~~ Upgrade Operator to 3.0.1 Jul 25, 2020

harshavardhana merged commit bc8429b into minio:master Jul 25, 2020

Upgrade Operator to 3.0.1 #201

Upgrade Operator to 3.0.1 #201

Uh oh!

Conversation

dvaldivia commented Jul 24, 2020

Uh oh!

bexsoft left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cesnietor commented Jul 24, 2020

Uh oh!

Uh oh!

dvaldivia commented Jul 24, 2020

Uh oh!

cesnietor commented Jul 24, 2020

Uh oh!

dvaldivia commented Jul 24, 2020

Uh oh!

cesnietor left a comment

Choose a reason for hiding this comment

Uh oh!

dvaldivia commented Jul 24, 2020

Uh oh!

bexsoft commented Jul 24, 2020

Uh oh!

Uh oh!

cesnietor commented Jul 24, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cesnietor commented Jul 24, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cesnietor commented Jul 24, 2020

Uh oh!

cesnietor commented Jul 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bexsoft left a comment

Choose a reason for hiding this comment

Uh oh!

harshavardhana left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cesnietor commented Jul 25, 2020

Uh oh!

harshavardhana commented Jul 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cesnietor commented Jul 25, 2020

Uh oh!

harshavardhana commented Jul 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cesnietor commented Jul 25, 2020

Uh oh!

cesnietor left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

harshavardhana commented Jul 25, 2020

Uh oh!

cesnietor commented Jul 25, 2020

Uh oh!

Uh oh!

cesnietor commented Jul 24, 2020 •

edited

Loading

cesnietor commented Jul 24, 2020 •

edited

Loading

cesnietor commented Jul 25, 2020 •

edited

Loading

harshavardhana commented Jul 25, 2020 •

edited

Loading

harshavardhana commented Jul 25, 2020 •

edited

Loading

cesnietor left a comment •

edited

Loading