Did you forget the -r
when cloning a git repo with submodules? The command you’re looking for is git submodule update --init
GitLab YAML Docker Registry client
Have you written a Docker Registry API client in GitLab CI/CD YAML? I have.
# Delete candidate image from CI repository. clean-image: stage: .post except: - main variables: AUTH_API: "$CI_SERVER_URL/jwt/auth" SCOPE: "repository:$CI_PROJECT_PATH" REGISTRY_API: "https://$CI_REGISTRY/v2/$CI_PROJECT_PATH" before_script: - > which jq >/dev/null || (sudo apt-get update && sudo apt-get -y install jq) script: - echo "Deleting $CANDIDATE_IMAGE" - > TOKEN=$(curl -s -u "$CI_REGISTRY_USER:$CI_REGISTRY_PASSWORD" "$AUTH_API?service=container_registry&scope=$SCOPE:delete,pull" | jq -r .token) - > DIGEST=$(curl -s -I -H "Authorization: Bearer $TOKEN" -H "Accept: application/vnd.docker.distribution.manifest.v2+json" "$REGISTRY_API/manifests/$CI_COMMIT_SHORT_SHA" | tr -d "\r" | grep -i "^docker-content-digest: " | sed "s/^[^:]*: *//") - > curl -s -X DELETE -H "Authorization: Bearer $TOKEN" "$REGISTRY_API/manifests/"$(echo $DIGEST | sed "s/:/%3A/g")
sudo tee >/dev/null
Need to redirect to a file from sudo
? Use sudo tee >/dev/null
:
$ sudo ls -l /root >/root/files.list -bash: /root/files.list: Permission denied $ sudo ls -l /root | sudo tee /root/files.list >/dev/null $ sudo ls -l /root/files.list -rw-r--r-- 1 root root 94 Oct 18 09:31 /root/files.list
Use stdbuf to tee without buffering
Do you want to tee
the output of a command to a file, but see it in your terminal too, without buffering? The stdbuf
command can do this for you:
$ sudo stdbuf --output=L tcpdump -i any -tttt -n 'udp port 5353' | tee -a tcpdump-mdns
tcpdump: data link type LINUX_SLL2
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
2023-06-15 11:55:16.637670 eth0 M IP 10.0.0.23.5353 > 224.0.0.251.5353: 0 A (QM)? winnebago.local. (28)
2023-06-15 11:55:16.744660 eth0 M IP 10.0.0.42.5353 > 224.0.0.251.5353: 0*- [0q] 1/0/0 (Cache flush) A 10.0.0.42 (38)
...
Which uninstalled package provides a file?
$ apt-file find guestmount guestmount: /usr/bin/guestmount guestmount: /usr/share/bash-completion/completions/guestmount guestmount: /usr/share/doc/guestmount/changelog.Debian.gz guestmount: /usr/share/doc/guestmount/copyright guestmount: /usr/share/man/ja/man1/guestmount.1.gz guestmount: /usr/share/man/man1/guestmount.1.gz guestmount: /usr/share/man/uk/man1/guestmount.1.gz
Scripting DNS lookups
Are you writing a script and some command doesn’t accept hostnames and you don’t want to inline the IP address? dig +short
is your friend!
$ dig +short gbenson.net 69.163.152.201
Which APT repository did a package come from?
$ apt policy wget wget: Installed: 1.21.2-2ubuntu1 Candidate: 1.21.2-2ubuntu1 Version table: *** 1.21.2-2ubuntu1 500 500 http://gb.archive.ubuntu.com/ubuntu jammy/main amd64 Packages 100 /var/lib/dpkg/status
Python debugger
By the way, if you’ve never used Python’s debugger, it really is as simple as adding a call to the built-in function breakpoint()
at the point you want it to stop.
Flask on Elastic Beanstalk
I had a play with Elastic Beanstalk the other day. It’s one of those things people turn their noses up at, but it seems pretty good for prototyping and small things. My biggest issue so far has been that, for Python applications, it expects a WSGI callable called application
in the file application.py
… but I was using Flask, and every single Flask application ever has a WSGI callable called app
in the file app.py
. I tried to not care, but it got too much after about an hour so I went and found how to override it:
$ cat .ebextensions/01_wsgi.config option_settings: aws:elasticbeanstalk:container:python: WSGIPath: "app:app"
Thank you Nik Tomazic for that! (=⌒‿‿⌒=)
The other AWS IAM SSO problem
The other thing you’ll run into using IAM users in AWS CLI is that a lot of things don’t support SSO sessions anyway. If you configure an IAM user with an SSO session name as recommended you’ll get errors like this:
$ eb init -p python-3.8 eb-flask-app ERROR: InvalidConfigError - The profile "default" is configured to use SSO but is missing required configuration: sso_start_url, sso_region
and this:
$ terraform apply | Error: configuring Terraform AWS Provider: loading configuration: profile "default" is configured to use SSO but is missing required configuration: sso_region, sso_start_url
You can fix these by configuring without an SSO session:
$ aws configure sso SSO session name (Recommended): WARNING: Configuring using legacy format (e.g. without an SSO session). Consider re-running "configure sso" command and providing a session name. SSO start URL [None]: https://whatever.awsapps.com/start SSO region [None]: us-east-1 ...
You can also fix them by just editing your ~/.aws/config
, and copying the sso_start_url
and sso_region
keys from the [sso-session ...]
section into the relevant user’s section, but that might be a hack too far!