Amazon-web-services – kubectl : error: You must be logged in to the server (Unauthorized)

amazon-iamamazon-web-serviceskopskubectlkubernetes

I have created kops cluster and getting below error when logging to the cluster.

Error log :

*****INFO! KUBECONFIG env var set to /home/user/scripts/kube/kubeconfig.yaml
INFO! Testing kubectl connection....
error: You must be logged in to the server (Unauthorized)
ERROR! Test Failed, AWS role might not be recongized by cluster*****

Using script for iam-authentication and logged in to server with proper role before connecting.
I am able to login to other server which is in the same environment. tried with diff k8s version and diff configuration.

KUBECONFIG doesn't have any problem and same entry and token details like other cluster.
I can see the token with 'aws-iam-authenticator' command

Went through most of the articles and didn't helped

Best Answer

It seems as a AWS authorization issue. At cluster creation only the IAM user who created the cluster has admin rights on it, so you may need to add your own IAM User first.

1- Start by verifying the IAM user identity used implicitly in all commands: aws sts get-caller-identity

If your aws-cli is set correctly you will have an output similar to this:

{
    "UserId": "ABCDEFGHIJK",
    "Account": "12344455555",
    "Arn": "arn:aws:iam::1234577777:user/Toto"
}

we will refer to the value in Account as YOUR_AWS_ACCOUNT_ID in step 3. (in this example YOUR_AWS_ACCOUNT_ID="12344455555"

2- Once you have this identity you have to add it to AWS role binding to get EKS permissions.

3- You will need to edit the ConfigMap file used by kubectl to add your user kubectl edit -n kube-system configmap/aws-auth In the editor opened, create a username you want to use to refer to yourself using the cluster YOUR_USER_NAME (for simplicity you may use the same as your aws user name, example Toto in step 2) , you will need it in step 4, and use the aws account id (don't forget to keep the quotes ""),you found it in your identity info at step 1 YOUR_AWS_ACCOUNT_ID, as follows in sections mapUsers and mapAccounts.

  mapUsers: |
    - userarn: arn:aws:iam::111122223333:user/ops-user
      username: YOUR_USER_NAME
      groups:
        - system:masters
  mapAccounts: |
    - "YOUR_AWS_ACCOUNT_ID"

4- Finally you need to create a role binding on the kubernetes cluster for the user specified in the ConfigMap

kubectl create clusterrolebinding cluster-admin-binding \
    --clusterrole cluster-admin \
    --user YOUR_USER_NAME

Related Solutions

Kubectl error: You must be logged in to the server (Unauthorized)

It turns out that the tokens was invalid (not sure if it because of 12 hours expiration). If you simply F5 the browser page you didn't re-authenticated but still can access the console page, but actually the token should be updated by re-login ICP Portal again.

The issue was fixed by re-access the ICP portal:

https://<master host>:8443/console/

This will re-allow you authenticate. After that, go to admin -> configure client, paste the latest commands you will find the token might be updated. Executing the new commands solved the issue.

2 Question still left:

a) If the page was long opened and token expired, ICP portal page may not auto refreshed to force you re-login, that means the token in set-credentials command are still old.

b) Even setting old tokens are accepted and command never complain an error even warning. This may mislead us when token are changed on servers, e.g, If I saved the commands to a local txt file and re-execute it again (even after token expired), the commands still finished successful, but actually I still didn't get authenticated correctly when I try to login.

Amazon-web-services – kubectl error You must be logged in to the server (Unauthorized) when accessing EKS cluster

When an Amazon EKS cluster is created, the IAM entity (user or role) that creates the cluster is added to the Kubernetes RBAC authorization table as the administrator. Initially, only that IAM user can make calls to the Kubernetes API server using kubectl.

eks-docs

So to add access to other aws users, first you must edit ConfigMap to add an IAM user or role to an Amazon EKS cluster.

You can edit the ConfigMap file by executing: kubectl edit -n kube-system configmap/aws-auth, after which you will be granted with editor with which you map new users.

apiVersion: v1
data:
  mapRoles: |
    - rolearn: arn:aws:iam::555555555555:role/devel-worker-nodes-NodeInstanceRole-74RF4UBDUKL6
      username: system:node:{{EC2PrivateDNSName}}
      groups:
        - system:bootstrappers
        - system:nodes
  mapUsers: |
    - userarn: arn:aws:iam::111122223333:user/ops-user
      username: ops-user
      groups:
        - system:masters
  mapAccounts: |
    - "111122223333"

Mind the mapUsers where you're adding ops-user together with mapAccounts label which maps the AWS user account with a username on Kubernetes cluster.

However, no permissions are provided in RBAC by this action alone; you must still create role bindings in your cluster to provide these entities permissions.

As the amazon documentation(iam-docs) states you need to create a role binding on the kubernetes cluster for the user specified in the ConfigMap. You can do that by executing fallowing command (kub-docs):

kubectl create clusterrolebinding ops-user-cluster-admin-binding --clusterrole=cluster-admin --user=ops-user

which grants the cluster-admin ClusterRole to a user named ops-user across the entire cluster.

Best Answer

Related Solutions

Kubectl error: You must be logged in to the server (Unauthorized)

Amazon-web-services – kubectl error You must be logged in to the server (Unauthorized) when accessing EKS cluster

Related Topic