XARs: An efficient system for self-contained executables

https://code.fb.com/data-infrastructure/xars-a-more-efficient-open-source-system-for-self-contained-executables/

• XAR: eXecutable ARchive to distribute software
• Alternative: PEX = self-contained Python virtual environments
• 1 file
• highly compressed
• all deps included
• can be run from anywhere in the system
• Python, Lua applications, Node.js applications, C++ exe and data files
• Initially: PAR (Python Archive), that decompress to tmp folder, run in folder
• Problems: decompressing each time, cleanup
• Alternative: PEX: decompress once, reuse it
• XAR uses squashfs files (mount before use, unmount after idle timeout)
• “nearly zero overhead” in prod
• Benchmark:

  

• XAR:
  • 4kb preamble = shebang pointing to a helper executable
  • helper tool xarexec_fuse that knows how to read optionally mount, and execute content
  • FUSE filesystem squashfs with zstd compression

11 Ways (Not) to Get Hacked in k8s

https://kubernetes.io/blog/2018/07/18/11-ways-not-to-get-hacked/

• CONTROL PLAN
  • 1. TLS Everywhere (between every component on the master, between Kubelet and API server)

       

  • Autoscaling: we should scale k8s instances, but shouldn’t include tls key into the image
  • Kubelet TLS bootstrapping documentation (didn’t read): add low-permissions keys to generate a proper key, auto-approve.

    

  • 2. Enable RBAC with least privilege, disable ABAC, monitor logs
  • 3. Use Third-party Auth for API Server: Google, Github, Dex (OpenID Connect Identity), …
Single point to add/remove members, add/revoke permissions, 2FA, etc.
  • 4. Separate and Firewall your etcd Cluster
Ability to write to etcd = have root on entire cluster.
etcd should be on separate nodes, TLS certificates to work with it, firewall rules to allow access from API server only
  • 5. Rotate Encryption Keys. Some keys are rotated automatically (kubelet certs), but not all
• WORKLOADS
  • 6. Use Linux Security Features (SELinux, AppArmor, seccomp-bpf, …) and PodSecurityPolicies
PodSecurityPolicy = cluster-level resource, that controls security aspects of pod spec.
  • 7. Statically Analyze YAML (kubesec https://kubesec.io), run unit tests for k8s configs (https://github.com/garethr/kubetest)
Shift left testing concept https://en.wikipedia.org/wiki/Shift_left_testing
  • 8. Run containers as a Non-Root User. 
Run as a Root in the container => you have root permissions on host. Container breakouts is needed, but risk is higher.
  • 9. Use Network Policies (restrict all pod-to-pod traffic)
  • 10. Scan Images and Run IDS (Intrusion Detection Systems).
Admission controller gates - to check the Deployment/Pod/… Check content, mutate content.
A lot of open-source scanners that prevent releasing containers with known vulnerabilities
https://github.com/coreos/clair
https://github.com/aquasecurity/microscanner
Zero-day vulnerabilities - will alway exists, IDS should be added.
Detects unusual behaviours in a container, pause or kills it:
https://sysdig.com/product/secure/
• FUTURE
  • 11. Run a Service Mesh