Kaniko lets you build container images right inside Kubernetes, without a Docker daemon. In this post we’ll use it to create images and push them to an ECR repository.

The first thing to sort out is configuration: Kaniko needs the ECR URL and AWS credentials so it can talk to ECR through IAM. Its default config doesn’t ship with the AWS ECR helper tag, so we add our ECR FQDN and that tag to the file ourselves.

kanikoconfig.json

{
        "auths": {
        },
        "credHelpers": {
                "appengine.gcr.io": "gcr",
                "asia.gcr.io": "gcr",
                "eu.gcr.io": "gcr",
                "gcr.io": "gcr",
                "gcr.kubernetes.io": "gcr",
                "us.gcr.io": "gcr",
                "ACCOUNTNUMBER123.dkr.ecr.us-west-2.amazonaws.com":"ecr-login"
        }
}

To make this config available to the pod, we load it into a ConfigMap with kubectl:

$ kubectl create configmap credconfig --from-file=config.json 
configmap/credconfig created

On top of that config, Kaniko also needs AWS credentials, which we provide through kubectl create secret. To keep things least-privilege, I’m using IAM credentials scoped to ECR access only.

$ kubectl create secret generic aws-secret --from-file=credentials_only_ecr
secret/aws-secret created

With the config and credentials both in place, we can define the Kaniko pod that mounts them and runs the build.

Kaniko.yaml

apiVersion: v1
kind: Pod
metadata:
  name: kaniko
spec:
  containers:
  - name: kaniko
    image: gcr.io/kaniko-project/executor:latest
    args: ["--dockerfile=Dockerfile",
            "--context=s3://build-bucket-webischia/webischia.tar.gz",
            "--destination=ACCOUNTNUMBER123.dkr.ecr.us-west-2.amazonaws.com/kaniko-webischia:latest"]
    volumeMounts:
      - name: aws-secret
        mountPath: /root/.aws/
      - name: credconfig
        mountPath: /kaniko/.docker/
  restartPolicy: Never
  volumes:
    - name: aws-secret
      secret:
        secretName: aws-secret
    - name: credconfig
      configMap:
        name: credconfig

Once the pod runs, here are the Kaniko logs showing the build and push:

INFO[0000] Downloading base image python:3-slim         
ERROR: logging before flag.Parse: E1117 11:08:09.288305       1 metadata.go:142] while reading 'google-dockercfg' metadata: http status code: 404 while fetching url http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg
ERROR: logging before flag.Parse: E1117 11:08:09.292754       1 metadata.go:159] while reading 'google-dockercfg-url' metadata: http status code: 404 while fetching url http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg-url
2018/11/17 11:08:09 No matching credentials were found, falling back on anonymous
INFO[0001] Executing 0 build triggers                   
INFO[0001] Taking snapshot of full filesystem...        
INFO[0002] Skipping paths under /dev, as it is a whitelisted directory 
INFO[0002] Skipping paths under /etc/hosts, as it is a whitelisted directory 
INFO[0002] Skipping paths under /kaniko, as it is a whitelisted directory 
INFO[0002] Skipping paths under /proc, as it is a whitelisted directory 
INFO[0002] Skipping paths under /root/.aws, as it is a whitelisted directory 
INFO[0002] Skipping paths under /sys, as it is a whitelisted directory 
INFO[0002] Skipping paths under /var/run, as it is a whitelisted directory 
INFO[0002] Using files from context: [/kaniko/buildcontext] 
INFO[0002] ADD . /app                                   
INFO[0002] Taking snapshot of files...                  
INFO[0002] WORKDIR /app                                 
INFO[0002] cmd: workdir                                 
INFO[0002] Changed working directory to /app            
INFO[0002] Taking snapshot of full filesystem...        
INFO[0002] Skipping paths under /dev, as it is a whitelisted directory 
INFO[0002] Skipping paths under /etc/hosts, as it is a whitelisted directory 
INFO[0002] Skipping paths under /kaniko, as it is a whitelisted directory 
INFO[0002] Skipping paths under /proc, as it is a whitelisted directory 
INFO[0002] Skipping paths under /root/.aws, as it is a whitelisted directory 
INFO[0002] Skipping paths under /sys, as it is a whitelisted directory 
INFO[0002] Skipping paths under /var/run, as it is a whitelisted directory 
INFO[0002] No files were changed, appending empty layer to config. No layer added to image. 
INFO[0002] CMD ["python3","fahri.py"]                   
2018/11/17 11:08:14 pushed blob sha256:d0eb458241dea91234213a46893b0fe6cec42576ed3ea9b283b988bc97067ac32
2018/11/17 11:08:14 pushed blob sha256:ba03597532d52402745666b266c03983149de129de5ca8eb01ec321412390ac6
2018/11/17 11:08:15 pushed blob sha256:3a6fba0409825ce0c720161f7c30e12faad5f210bf1ebffa28df16ed112958a6
2018/11/17 11:08:18 pushed blob sha256:a5a6f2f73cd8abbdc55d0df0d8834f7262713e87d6c8800ea3851f103025e0f0
2018/11/17 11:08:18 pushed blob sha256:a4b11c375c52d7c9d772123475d2f6e5784b0e7b249a47f123423b8312f53002
2018/11/17 11:08:19 pushed blob sha256:02c57c00f1bc8cdead0eee841a09277827b45c91e3ff1f550bc95d2823fe1934
2018/11/17 11:08:22 pushed blob sha256:738eb12343412bff2050fa1e0b87f07212341272028fc82c61923412efc0c435
2018/11/17 11:08:22 ACCOUNTNUMBER123.dkr.ecr.us-west-2.amazonaws.com/kaniko-webischia:latest: digest: sha256:f0513659f2492d30efcbdd0d0e2814da5f8aa4922bc833591c5ab5858c16f036 size: 1242

To confirm the image really landed, let’s check the ECR repository:

$ aws ecr describe-images --repository-name kaniko-webischia
{
    "imageDetails": [
        {
            "imageSizeInBytes": 50857532, 
            "imageDigest": "sha256:f0513659f2492d30efcbdd0d0e2814da5f8aa4922bc833591c5ab5858c16f036", 
            "imageTags": [
                "latest"
            ], 
            "registryId": "ACCOUNTNUMBER123", 
            "repositoryName": "kaniko-webischia", 
            "imagePushedAt": 1542452903.0
        }
    ]
}