Each system can produce data, consume data or both.

Integrating between systems becomes a must.

If system A produces data, and System B & C needs this data, you will have to integrate A +B , then A+C.

Integrating systems costs a lot with this way ( i mean two by two).

- Imagine you have 4 tools, the number of possible integration between every two is : C(4,2) = 6

- Imagine you have 10 tools, the number of possible integration between every two is : C(10,2) = 45

Note: C(n,k) is a maths formula, check it here: https://www.calculatorsoup.com/calculators/discretemathematics/combinations.php

Kafka comes to avoid all these integration. Instead of integrating two by two, you just need to put Kafka as centralized place for Data Streaming : Data producers ( systems which produces data) need to report their data to Kafka & Data consumers ( systems which needs this data) they just need to subscribe to Kafka ( e.g …

There are two ways:

In json format (but very long output)

ceph osd cursh dump

Or in plain text format ( but easy to read)

ceph osd getcrushmap | crushtool -d -

• getcrushmap will return binary content
• crushtool will convert it to plain text

we recommend to use Gluster native client which is “glusterfs-fuse”. However, if you still insist to use NFS protocol, the article is written for that purpose.

Solution

Use nfs-ganesha

Solution Overview

• Install GlusterFS
• Then, install NFSv4 server specific for Gluster Storage
• enable firewall of 2 services: nfs and rpc-bind

There are a lot of ways to secure a container image for runtime :

1. Inherit from a distroless image base
2. Default user of image is not-root ( USER 1001)
3. If the image expose a port, it must be above 1024 because ports under 1024 requires root.
4. Files used by the main process must be owned by random user and root group (chown 1001:0)

I search on a Nginx image that complies with these criteria,

I found 2 images however .. :

• docker.io/kyos0109/nginx-distroless : which focus only on the 1st way.
• docker.io/nginxinc/docker-nginx-unprivileged: which focus on all ways except the first.

As consequence, I built a new image which leverages these two images and applies all security ways. …

Spending a lot of time to find the best memory-optimized instance but also it should have amd64 as architecture because the current version of EKS is not well compatible with arm64 cpu architecture.

The happy day came when i visited the home page of this software :

ec2-instance-selector  --memory 16  --vcpus 4 --cpu-architecture amd64  -r ap-southeast-1

Amazing !

install yq CLI then :

yq r application.yaml --printMode pv "**" | sed 's/: /=/' > application.properties

Details ? check ; https://stackoverflow.com/a/63098014/747579

Done!

Thanks

alias ab='kubectl run test-load --rm --tty -i --restart='Never' --image devth/alpine-bench --command -- /go/bin/main'ab -n 10000 -c 900 -s 300 https://my.site123.com/

Related Github Git

Overview

May 28 2020, It was a hard day. Indeed. The famous public container registry, quay.io , was down for several hours.

The same day, we decided to upgrade the EKS Kubernetes cluster from 1.14 to 1.15

While rolling out the upgrade by terminating some worker nodes and let the autoscaling group spin off new nodes, the incident occurs.

Indeed, the Main router of all user requests ( Ingress Controller) cannot run on any node with the error ImagePullBackOff

Root Cause

After some investigation, I realized that all pods with images from quay.io are not downloadable.

I’ve checked quay.io , and it was the disaster: quay.io …