Neither one nor Many
Software engineering blog about my projects, geometry, visualization and music.
This is just to have some list somewhere, sorted from newer to old.
Work in progress..
This was a lightningtalk I prepared.
https://cdn.cppse.nl/lightningtalk2.pdf
This was a lightningtalk I prepared.
https://cdn.cppse.nl/lightningtalk.pdf
Since I went to a DDD / Event storming workshop decided to summarize some learnings and present to collegues. https://cdn.cppse.nl/workshopddd.pdf
Created this talk more for collegues that wanted to become more familiar with the linux shell.
This is mostly for keeping track of what equipment I have, and to keep notes regarding my drones, or drones in general.
A few years ago I got interested in soldering, later in electronics, later in arduinos, then 3D printing, and now a new hobby that is made possible thanks to all this.
It took a while to learn the basics, and now I finally reached a level where I can at least compose my own drones and build them from scratch.
This whole mess started with ordering parts for: https://www.thingiverse.com/thing:3249126 I pretty much stuck with all the recommended parts, see https://blog.prusaprinters.org/how-to-build-a-3d-printed-micro-drone/
Flight controller + ESC board | HGLRC XJB F413 Omnibus F4 Flight Controller & 13A Blheli_S 2-3S... | url | 52.71 EUR |
Motors | Racerstar Racing Edition 1103 BR1103B 8000KV 10000KV 1-3S... | url | 7.43 EUR |
Propellers | 5 Pairs Eachine 2035 50mm 4 Blade Propellers ABS For Eachine... | url | 3.21 EUR |
Battery | Giant Power 7.4V 300mAh 2S 35C Battery For HCP100s T-REX 150 F3P | url | 5.33 EUR |
Battery charger | Charsoon Lipo Charger | url | 12.50 EUR |
Radio Transmitter | FlySky FS-i6 2.4G 6CH AFHDS RC Radion Transmitter With FS-iA6B... | url | 38.42 EUR |
Flysky Receiver | FS-RX2A Pro V1 2.4G Compatible Receiver for FS-I6 FS-I6X FS-I6S... | url | 7.35 EUR |
Connector for battery | 5 Pairs 100mm 10cm JST Male And Female Plug Connector Cable ESC... | url | 1.49 EUR |
Connect receiver with flight controller | Foxeer XAT600M 4Pin Servo Cable | url | 1.02 EUR |
Frame | 3D printed | - | |
FPV goggles | Eachine VR D2 Pro 5 Inches 800*480 40CH 5.8G Diversity FPV... | url | 66.13 EUR |
FPV camera + built in transmitter | Eachine TX05 0.01/5/25/50/100/250mW Switchable w/ OSD AIO 5.8G... | url | 23.22 EUR |
Almost ready to fly wizard x220 | Eachine Wizard X220 FPV Racing RC Drone Blheli_S F3 6DOF 2205... | url | 117.06 |
Receiver (already had it, came with transmitter) | Flysky 2.4G 6CH FS-iA6B Receiver PPM Output With iBus Port | url | 13.55 EUR |
3S batteries used most | ZOP Power 3S 11.1V 1500MAH 25C Battery XT60 Plug | url | 11.54 EUR |
FPV Camera (upgrade) | caddx turtle v2 (special lens) | url | 63.95 EUR |
VTX (upgrade) | Eachine TS5828L Micro 5.8G 600mW 40CH Mini FPV Transmitter VTX... | url | 8.79 EUR |
Antenna (upgrade) | AOMWAY 5.8G 3dBi RP-SMA Male Four Lobe RHCP VTx/VRx FPV Antenna | url | 10.75 EUR |
Custom camera mount | 3D designed + printed | - |
Stock propellers broke too easily, you could see the plastic get "tired" quickly. The DAL props really where indestructable compared to the stocks.
Tri blades | DAL Cyclone T5045C Propellers (4 stuks) - BLAUW | url | 2.49 EUR |
Bi blades | DAL 5045 Indestructible Bullnose Propellers - Groen (4 stuks) | url | 1.95 EUR |
3S | WLtoys V950 RC Helicopter Part 1500mAh 11.1V 20C 3S T Plug Lipo... | url | 16.42 EUR |
4S | Tattu LiPo accupack 14.8 V 1300 mAh Aantal cellen: 4 45 C Softcase XT60 | url | 20.49 |
Only flew the 4S battery once, which was when I lost the drone
Replacement ESC (one burned probably shorted with the frame) | Aikon SEFM 20A 2-4S ESC | url | 12.95 EUR |
This ESC after configuring the same settings as the three remaining stock ESC's worked perfectly together, even though it's a different one. The quad flew perfectly fine.
Frustrated with failures trying to build the TinyTina, trying different motors, different flight controller, it just didn't work well with the 3D printed frame anymore. It flew, but then it would bend and the props would hit something. So I looked for a more sturdy frame.
Frame | Awesome E90 90mm Micro Brushless Frame Kit Carbon Fiber 12g RC Drone FPV Racing Frame Multi Rotor | url | 10.32 EUR |
Motors | Racerstar Racing Edition 1103 BR1103B 8000KV 10000KV 1-3S Brushless Motor Black For 50-100 RC Drone FPV Racing - 8000KV | url | 7.34 EUR |
Flight controller | Aikon F42020 Flight Controller | url | 32.95 EUR |
4-in-1 ESC | AIKON AK32PIN 25A 2-6S 4-IN-1 ESC | url | 53.95 EUR |
The following parts are identical to the TinyTina build.
Flysky Receiver | FS-RX2A Pro V1 2.4G Compatible Receiver for FS-I6 FS-I6X FS-I6S... | url | 7.35 EUR |
Connector for battery | 5 Pairs 100mm 10cm JST Male And Female Plug Connector Cable ESC... | url | 1.49 EUR |
Propellers | 5 Pairs Eachine 2035 50mm 4 Blade Propellers ABS For Eachine... | url | 3.21 EUR |
FPV camera + built in transmitter | Eachine TX05 0.01/5/25/50/100/250mW Switchable w/ OSD AIO 5.8G... | url | 23.22 EUR |
Different batteries though, the battery wasn't powerful enough. Bought a few 450mAh with 20C, 45C. Turns out the 45C was the only one that was able to lift it off the ground. Now I did some redesigns and it became heavier, and now it won't come off the ground again with 45C, also not with an 80C 450mAh 2S battery. I have to troubleshoot the bottleneck now, either the props should be bigger (plenty of space, since I moved the entire stack below the frame), or the battery needs to be 3S, OR the capacitor has to go (I think it's recommended for 6S usually anyway, and maybe the soldering joints for the battery are not perfect).
Batteries tried
Conrad energy LiPo accupack 7.4 V 450 mAh Aantal cellen: 2 40 C Softcase BEC-bus | url | 9.55 EUR | |
Gens ace LiPo accupack 7.4 V 450 mAh Aantal cellen: 2 25 C Softcase BEC | url | 7.39 EUR | |
Conrad energy LiPo accupack 7.4 V 350 mAh Aantal cellen: 2 25 C Softcase BEC-bus | url | 4.77 EUR | |
5Pcs Gaoneng GNB 7.4V 450mAh 2S 80/160C Lipo Battery JST Plug For Eachine Aurora 90 100 FPV Racer | url | 27.88 EUR |
Remapping motors
https://github.com/betaflight/betaflight/wiki/Remapping-Motors-with-Resource-Command-(3.1)
Long m2 screws
https://www.amazon.co.uk/sourcingmap-M2x30mm-Socket-Knurled-Screws/dp/B015A31EVK/ref=sr_1_7?keywords=m2+30mm&qid=1563786654&s=gateway&sr=8-7
In cli
resource list
resource motor 1 a7 resource motor 2 a12 resource motor 3 a6 resource motor 4 a11 save
https://github.com/betaflight/betaflight/wiki/Remapping-Motors-with-Resource-Command-(3.1)
Source: https://blog.alexellis.io/golang-writing-unit-tests/
go test
go test -cover
go test -cover -coverprofile=c.out go tool cover -html=c.out -o coverage.html
If you have main.go
, add main_test.go
:
package main
import "testing"
func TestSum(t *testing.T) {
total := Sum(5, 5)
if total != 10 {
t.Errorf("Sum was incorrect, got: %d, want: %d.", total, 10)
}
}
More info here: https://golang.org/pkg/testing/
Source: https://blog.golang.org/profiling-go-programs (by Ross Cox)
var cpuprofile = flag.String("cpuprofile", "", "write cpu profile to file")
func main() {
flag.Parse()
if *cpuprofile != "" {
f, err := os.Create(*cpuprofile)
if err != nil {
log.Fatal(err)
}
pprof.StartCPUProfile(f)
defer pprof.StopCPUProfile()
}
Build an capture profiling data
go build && ./myprogram -cpuprofile=my.prof
go tool pprof myprogram my.prof
Commands available are:
top5
top10
top10 -cum
list main
list myfoobar.function.func1
There are also very similar flags for memory profiling. I didn't try this myself yet, check the blog I got this from in the first place.
kubectl get node -o go-template='{{printf "%-40s %20s %20s\n" "NODE" "CAPACITY" "ALLOCATABLE"}}{{range $index,$pod := .items}}{{$capacity := index .status.capacity "nvidia.com/gpu"}}{{$alloc := index .status.allocatable "nvidia.com/gpu"}}{{printf "%-40s %20s %20s\n" .metadata.name $capacity $alloc}}{{end}}'
kubectl api-resources --verbs=list --namespaced -o name
# useful for debugging
kubectl api-resources --verbs=list --namespaced -o name | xargs -n 1 -I{} /bin/sh -c "kubectl get {} -o yaml || true; kubectl get {} --all-namespaces -o yaml || true"| tee out.log
kubectl get po -o json | jq .items[].status.podIP -r | fping
kubectl get namespace "foo-restricted" -o json \
| tr -d "\n" | sed "s/\"finalizers\": \[[^]]\+\]/\"finalizers\": []/" \
| kubectl replace --raw /api/v1/namespaces/foo-restricted/finalize -f -
kubectl run -i --tty busybox --image=busybox:latest --restart Never --rm -n $NAMESPACE -- sh
kubectl get pod -A
kubectl get pod --show-labels
kubectl get pod -l foo=bar,baz=ban
No time to waste?
kubectl delete pod --grace-period=0 --force
Cleanup all terminating pods
kubectl get pod|grep Terminating|awk '{print $1}'|xargs -n 1 echo kubectl delete pod --grace-period=0 --force
Cleanup all non-running pods
kubectl get pods --field-selector=status.phase!=Running -o jsonpath="{range .items[?(@.status.phase != 'Running')]}kubectl delete pod -n {.metadata.namespace}{' '}{.metadata.name}{'\n'}{end}" | sh -
kubectl get pod -A --no-headers |grep -v Running | awk '{print "kubectl delete pod -n "$1" "$2}'|sh -
This script has helped me so many times quickly finding issues somewhere in k8s containers.
#!/bin/bash
function cleanup
{
kill -9 $(jobs -p)
}
trap "cleanup" 2
function prefix
{
while read line; do
echo "$1$line";
done < /dev/stdin
}
function kubectl_log
{
typeset namespace="$1"
typeset pod="$2"
shift 2
for i in $(seq 1 $#); do
typeset container="$1"
shift
kubectl logs -n $namespace $pod -f $container | prefix "[ $namespace - $pod - $container ] - " &
done
}
while read line; do
kubectl_log $line
done < <(kubectl get pods -A -o jsonpath="{range .items[*]}{.metadata.namespace} {.metadata.name} {..containerStatuses..name}{'\n'}{end}")
wait
#!/bin/bash
function prefix
{
while read line; do
echo "$1$line";
done < /dev/stdin
}
echo "----------------------------"
echo checking direct mounts
echo "----------------------------"
while read ns; do
if [[ "$ns" == "kube-system" ]]; then
continue
fi
kubectl get pod -n $ns -o jsonpath="{range ..volumes[?(@.hostPath.path != '')]}{.hostPath.path}{'\n'}{end}" | sort | uniq | prefix "$ns - "
done < <(kubectl get ns -o jsonpath="{range .items[*]}{.metadata.name}{'\n'}{end}")
echo "----------------------------"
echo checking persistent volumes $ns
echo "----------------------------"
kubectl get pv -o yaml -o jsonpath="{range .items[*]}{..hostPath.path}{'\n'}{end}" | sort | uniq
kubeadm token create --print-join-command
kubectl auth can-i --list --namespace=test-restricted
[root@nas kube]# kubectl get no Unable to connect to the server: x509: certificate is valid for 10.96.0.1, 192.168.2.2, not 10.75.99.139
[root@nas kube]# kubectl --insecure-skip-tls-verify get no ...
root@node001:~> kubectl describe ep -n kube-system kube-controller-manager
Name: kube-controller-manager
Namespace: kube-system
Labels: <none>
Annotations: control-plane.alpha.kubernetes.io/leader:
{"holderIdentity":"rb-haha-b_35d43390-6650-4907-99c5-57d1e6455496","leaseDurationSeconds":15,"acquireTime":"2021-01-26T13:46:55Z","renewTi...
Subsets:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal LeaderElection 17m kube-controller-manager rb-haha-b_35d43390-6650-4907-99c5-57d1e6455496 became leader
curl -k https://<IP>:5000/v2/_catalog | jq .
docker save myimage:latest | gzip > myimage_latest.tar.gz
docker load < myimage_latest.tar.gz
Potentiometer example / Analog reading/writing / mapping
const int analogInPin = A0; // Analog input pin that the potentiometer is attached to const int analogOutPin = 9; // Analog output pin that the LED is attached to int sensorValue = 0; // value read from the pot int outputValue = 0; // value output to the PWM (analog out) sensorValue = analogRead(analogInPin); outputValue = map(sensorValue, 0, 1023, 0, 255); analogWrite(analogOutPin, outputValue);
Some electronics notes
L7805CV - 5V - Voltage regulator
Ultra-Low Quiescent Current LDO Regulator
IRFZ44N Mosfet
Not a "logic-level" mosfet, so I bought the wrong thing,
Source: http://forum.arduino.cc/index.php?topic=74718.0
"If you want to drive this from an Arduino, which only outputs 5V, you will need a "logic-level" MOSFET. For this type of MOSFET, Vgs=5V is enough to turn it on. For example, the ST STP55NF06L would be appropriate and is comparable to the IRFZ44N, except the STP55NF06L is logic-level.
You should also have a resistor in series with the Arduino output to limit the current, since the MOSFET gate is highly capacitive and can draw a big instantaneous current when you try to turn it on (or off). 220 ohms or so is appropriate."
Logic-level alternatives: * FQP30N06L * STP55NF06
Difference between N or P-channel mosfet:
MOSFETs come in two polarities, P channel and N channel, where "P" stands for positive and "N" stands for negative.
P Channel To turn a P channel MOSFET on, you apply a negative voltage to the gate. This voltage is negative relative to ground. In a circuit, you connect the P channel MOSFET's source terminal to a positive voltage supply and the drain to a resistor connected to ground; the resistor limits the current flowing through the transistor. The circuit diagram symbol for a P channel MOSFET has an arrow pointing away from the gate.
N Channel An N channel MOSFET turns on when you apply a positive voltage at its gate terminal. The voltage is greater than the positive voltage supply at the drain terminal. A resistor between the positive supply and the drain limits current; for an N channel MOSFET, the source terminal connects to ground. The circuit symbol for an N channel MOSFET has an arrow pointing toward the device's gate.
DHT22 - Temperature and Humidity sensor.
Info on the part: https://create.arduino.cc/projecthub/attari/temperature-monitoring-with-dht22-arduino-15b013
ESP8266, wemos d1 arduino studio + DHT22: https://www.losant.com/blog/getting-started-with-the-esp8266-and-dht22-sensor
NOTE!!!! However, be careful hooking it up to 5V and then to an input pin on the ESP8266!!! I think I may have bricked one because of that. It seems to work for a while, but after some time I couldn't program the thing anymore. NOTE: It's unclear if it supports 5V imo: https://hackaday.com/2016/07/28/ask-hackaday-is-the-esp8266-5v-tolerant/ also: https://forum.arduino.cc/index.php?topic=428521.0
Also found you should use a 10k resistor as a pull-up here: https://odd-one-out.serek.eu/projects/esp8266-nodemcu-dht22-custom-modules-firmware/
3.3V voltage regulators
TODO: https://www.sparkfun.com/products/526
PIR 5Pcs/Lot SR501 HC-SR501 Adjust IR Pyroelectric Infrared PIR module Motion Sensor Detector Module for Arduino
https://nl.aliexpress.com/item/5PCS-HC-SR501-Adjust-IR-Pyroelectric-Infrared-PIR-Motion-Sensor-Detector-Module/32644972192.html
ESP8266
upload issues
https://forum.arduino.cc/index.php?topic=480507.0 https://github.com/esp8266/Arduino/blob/master/doc/faq/a01-espcomm_sync-failed.rst https://arduino.stackexchange.com/questions/41847/esp-wemos-d1-upload-fails
import code
code.interact(local=locals())
Or using IPython:
import IPython
IPython.embed()
name params with ?P<X>
,'
In [18]: output = "x:3372 y:1280 screen:0 window:29360137"
In [19]: reg = re.compile(r'x:(?P<X>\d*) y:(?P<Y>\d*)')
In [20]: reg.match(output).group('X')
Out[20]: '3372'
match non-greedy by adding ?
suffix
import json
from pprint import pprint
with open('data.json') as f:
data = json.load(f)
pprint(data)
Edit for Python3:
with open('data.json', encoding='utf-8') as data_file:
data = json.loads(data_file.read())
There is a plugin for pycharm. Installed it, and also pip install yapf
.
Then mv /usr/local/bin/yapf /usr/local/bin/yapf.x
and created this bash script to tweak some values:
#!/bin/bash
/usr/local/bin/yapf.x --style='{based_on_style: google, column_limit: 100}' "$@"
file = open("testfile.txt","w")
file.write("Hello World")
file.close()
import tempfile
tf = tempfile.NamedTemporaryFile()
temp_file_name = tf.name
tf.close()
or better:
import tempfile
with tempfile.NamedTemporaryFile(dir='/tmp', delete=False) as config:
# temp_file_name = config.name
config.write('sdfkjskldf')
with exec_helpers:
with exec_helpers.Subprocess() as executor:
apiserver = executor.check_call('module load kubernetes && kubectl config view -o ' \
'jsonpath="{.clusters[0].cluster.server}"')
if apiserver:
return apiserver.stdout_str
raise AbortException("Could not retrieve the Kubernetes API server address")
# ret.exit_code == 0
without:
subprocess.check_output(["echo", "Hello World!"])
subprocess.check_output("exit 1", shell=True)
subprocess.check_output("ls non_existent_file; exit 0", stderr=subprocess.STDOUT, shell=True)
For in base class:
import abc
@abc.abstractmethod
def init(self):
return
import os
import stat
st = os.stat('somefile')
os.chmod('somefile', st.st_mode | stat.S_IEXEC)
import textwrap
textwrap.dedent('''\
#!/bin/bash
set -ex
sleep 10
echo Failure is the mother of success.
''')
str()
-> !s
repr
-> !r
f"kubectl still returned data for namespace: {json!s}, expected NotFoundException"
f"kubectl still returned data for namespace: {json!r}, expected NotFoundException"
pip install pyyaml
1 import yaml 2 3 with open("/etc/kubernetes/kubelet.kubeconfig") as f: 4 p = yaml.load(f) 5 print(p['clusters'][0]['name'])
from datetime import datetime
datetime.strftime(datetime.now(), "%Y-%m-%d %H:%M:%S")
Place to keep commands I tend to google more than once for some reason.
Start cmd.exe as Administrator
powercfg.exe /hibernate off|on
Some file in C:\ hiberfil.sys (IIRC) will consume quite a bit of disk space which may be a reason for not using the hibernate feature.
Assuming it's already flashed etc., (I forgot the steps because it's too long ago!)
What I do is connect the Edison via both micro USB's, one will be used as power, the other can be used for connecting to it.
In case the edison is booted you can open the terminal with screen
:
sudo screen /dev/ttyUSB0 115200 115200
Configure the WiFi via configure_edison --wifi
One caveat if you are also on the same network (i.e., your laptop is also on the same network) you may get a conflict via the USB wire.
You can see with ifconfig
two IP addresses one for the wlan0
and one for usb0
, which is probably causing a conflict somehow.
Solution is:
ifconfig usb0 down
Some stuff I keep here as a cheetsheat for myself
NOTE: do not run in case you have .git
directory, it can corrupt it.
Source: https://stackoverflow.com/a/1583282/1958831
mv .git /tmp/git_tmp
find . -type f -print0 | xargs -0 sed -i 's/subdomainA\.example\.com/subdomainB.example.com/g'
mv /tmp/git_tmp .git
$?
return last exit code
stdbuf
time make -j24 "$@" 2>&1 | stdbuf -i0 -o0 -e0 tee ./mm.log
sets buffer to zero, everything is piped immediately, bad for throughput, good for latency
gdb
on program that works on stream, with mkfifo
to make this a combined example (you can issue the bash command instead of the tail -f
in this case)
mkfifo /tmp/foo
bash tail_all.sh > /tmp/foo
gdb <application>
run <program args> < <(tail -f /tmp/foo)
This syntax is also useful for stuff like:
while read -d ':' line; do
echo "$line"
done < <(hbase classpath)
A pipe |
creates a new subshell, this is also a nice way to avoid this (subshells cannot modify variables of their parent)!
EDIT: This is called process substitution, see http://tldp.org/LDP/abs/html/process-sub.html
cat <<EOT >> greetings.txt
line 1
line 2
EOT
if [[ $foo =~ foo ]]; then
;
done
Note that you DO NOT use asterisks like *foo*
!
stdin
A pattern I use quite a bit is the following.
function func
{
while read line; do
echo $line
done < /dev/stdin
}
cat foo.txt | func
Sometimes subshells are useful.
echo mypid = $!
( sleep 10; ) &
typeset pid=$!
sleep 1
kill $pid # kill if still running..
Note that you can use exit
in a subshell without destroying the parent.
function myfunc
{
echo Do some cleanup here..
exit 0
}
trap "myfunc" 2
print "Press CTRL+C to abort 10 seconds of waiting.."
sleep 10
First learned about this in kornshell, but seems to work for bash as well.
function func {
local -n test_ref=$1
test_ref="Hello world"
}
func test # note omitted $-sign
echo $test
If you want kornshell compatibility use typeset -n
instead of local -n
.
The same example, but with an array
, you have to first explicitly declare the array before passing it to func
.
declare -A array
function func {
local -n array_ref=$1
array_ref=([one]=aaa [two]=bbb [three]=ccc)
}
func array
(If you want kornshell compatibility again, use typeset -A
instead of declare -A
(and typeset -n
for local -n
)).
array
sContinuing the previous paragraph, some useful examples regarding array
s.
echo ${array[@]} # aaa bbb ccc
echo ${!array[@]} # one two three
echo ${#array[*]} # 3
for key in ${!array[@]}; do
local value=${array[$key]}
echo $key = $value # one = aaa, ...
done
Instead of initializing with the array=([foo]=bar [baz]=ban)
syntax, assignment syntax is array[foo]=bar
.
typeset hostname=$(hostname -s)
hostname=${hostname:0:10} # first 10 chars..
Trick to use cachefilesd
on NFS mounts. Useful if you work remote on a mounted NFS share.
btrfs
is not supported, so I create an ext4
mount instead like this:
dd if=/dev/zero of=/tmp/fscache.txt bs=1M count=4000
# be careful with choosing /dev/loopN
losetup /dev/loop10 /tmp/fscache.txt
mkfs.ext4 /dev/loop10
mount /dev/loop10 /var/cache/fscache
Very useful other commands can be found here: http://www.cyberciti.biz/faq/centos-redhat-install-configure-cachefilesd-for-nfs/
(i.e.: cat /proc/fs/nfsfs/*
)
start iperf -s
server, then iperf -c 127.0.0.1
.
atop
can do this, which comes provided by your package manager probably, but chances are you need to compile it yourself for a newer version.
The new version makes it possible to compile a kernel module, which (after reboot) will make network counters available in atop
. Quite awesome!
export interface=enx9cebe8349fdc
sudo tc qdisc add dev $interface root netem delay 10000ms
sudo tc qdisc del dev $interface root netem
Prints files count per directory for current directory level:
du -a | cut -d/ -f2 | sort | uniq -c | sort -nr
http://superuser.com/questions/53103/udp-traffic-through-ssh-tunnel https://securesocketfunneling.github.io/ssf/#download
iptables
to redirect ip addressiptables -t nat -A OUTPUT -d 10.141.0.1 -j DNAT --to-destination 8.8.8.8
http://superuser.com/questions/681705/using-iptables-to-redirect-ip-address
http://www.rutschle.net/tech/sslh.shtml i.e., http://www.rutschle.net/sslh#using-proxytunnel-with-sslh
WHAT IS IT? sslh accepts connections on specified ports, and forwards them further based on tests performed on the first data packet sent by the remote client.
trigen@zenbook:~> echo "Hello world (y)" | sed 's/(y)/:-)/'
Hello world :-)
trigen@zenbook:~> echo "Hello world (y)" | sed 's/.*\((y)\)/smiley: \1/'
smiley: (y)
trigen@zenbook:~> sed -i.bak 's/H/J/g' test.txt
trigen@zenbook:~> cat test.txt
Jello world
trigen@zenbook:~> cat test.txt.bak
Hello world
Another nice example
sed -i '/upload_max_filesize/s/=.*/= 100M/' /etc/php/7.2/apache2/php.ini
(-n don't echo all (double check if this is true), /p to print, \1 to print between ()'s (escaped))
fdisk -l |& sed -n 's/^Disk \/\([^:]*\).*$/\/\1/p'
fdisk -l |& sed -n 's/^Disk \/\([^:]*\).*$/\/\1/p' | xargs -n 1 -I{} /bin/sh -c "echo {}; smartctl -H {}"
Nice resource for sed
stuff: https://www.grymoire.com/Unix/Sed.html#uh-9
nc -z -v -w1 cppse.nl 80
openssl x509 -in <INPUT> -text
openssl s_client -showcerts -connect cppse.nl:443 </dev/null
# in case of SNI (combined domains in certificate)
openssl s_client -showcerts -servername cppse.nl -connect www.cppse.nl:443 </dev/null
# grep for alt subjects
openssl s_client -connect 10.141.0.1:10443 -servername 10.141.0.1 < /dev/null 2>/dev/null | openssl x509 -noout -text | grep -A1 'Subject Alternative Name'
# or..
openssl s_client -showcerts -connect 10.3.195.44:10443 </dev/null | openssl x509 -noout -ext subjectAltName
source: https://www.systutorials.com/239880/change-systemd-boot-target-linux/
systemctl enable multi-user
systemctl set-default multi-user
systemctl enable graphical
systemctl set-default graphical
systemctl list-units --type target --state active
echo '{"foo": "lorem", "bar": "ipsum"}' | python -m json.tool
openstack server list | grep ... | sed -n 's/.*\(10\.2[^;]*\);.*/\1/p'
kpartx -rav some.dd
readonly add partition devmappings verbose
mount /dev/mapper/loop0p5 /mnt/loop0p5/
umount /mnt/loop0p5
kpartx -d some.dd or kpartx -d /dev/loop0
see losetup -a for available loops
note losetup -D doesn't work for some reason when the partitions were created with kpartx
source: https://unix.stackexchange.com/questions/26743/force-gnu-screen-to-reflow-to-current-terminal-width-on-reattach?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa
"after you reattach a ctrl-a F runs the "fit" command to resize the current window. if you reattach using the -A option it should resize all windows when you reattach. "
Assuming the hash of the commit you want is c5f567 (source):
git checkout c5f567 -- file1/to/restore file2/to/restore
The git checkout man page gives more information. If you want to revert to the commit before c5f567, append ~1 (works with any number):
git checkout c5f567~1 -- file1/to/restore file2/to/restore
Taken from source:
git log --full-history -- [file path]
git log --full-history -1 -- [file path]
After some additional research, ar -t can be used to enumerate the object files in an archive, so after that it's just a matter of providing that list to ar as you usually would when creating an archive.
The following script handled this for all of the libraries at once:
for lib in `find -name '*.a'`;
do ar -t $lib | xargs ar rvs $lib.new && mv -v $lib.new $lib;
done
There is a difference in clipboards I've noticed, which confuses me bigtime.
Apparently one is the "X clipboard":
echo Hello world | xclip
Whenever pasting doesn't work try the middle mouse button..
Or use xsel
instead:
echo Hello world | xsel --clipboard
echo Hello world | xsel -b # same
Some params for xsel
:
Selection options
-p, --primary Operate on the PRIMARY selection (default)
-s, --secondary Operate on the SECONDARY selection
-b, --clipboard Operate on the CLIPBOARD selection
find . -name '*.jpg' -execdir mogrify -resize 1024x {} \;
openssl s_client -showcerts -connect 10.2.61.184:8081
/usr/local/share/ca-certificates/
.pem
or they won't be recognized.sudo update-ca-certificates
should pick them upgit rebase -i HEAD~2
mark commits as edit, then use this over and over:
git commit --amend --author="Ray Burgemeestre <ray.burgemeestre@brightcomputing.com>"
git rebase --continue
socat TCP-LISTEN:8081,fork TCP:127.0.0.1:12345
TODO: https://stackoverflow.com/a/10216050/1958831
find ./ -name '*.meta' -type f -mtime -1;
# then delete
find ./ -name '*.meta' -type f -mtime -1 -exec rm -f {} \;
find ./var -printf "%p %TY-%Tm-%Td %TH:%TM:%TS %Tz\n" | grep 2022-01-01
source: https://sites.google.com/a/kossboss.com/main/linux---find-files-older-or-younger-then-certain-days---delete-them-too-if-you-want
source: https://ma.ttias.be/removing-a-package-without-its-dependencies-in-centos-or-rhel/
# rpm -qa | grep "php-sqlite2"
php-sqlite2-5.1.6-200705230937
# rpm -e --nodeps "php-sqlite2-5.1.6-200705230937"
Sometimes it's annoying, to reverse engineering rwx
-> 7, r-x
-> 5 and r-x
-> 5.
For example, I misread the group part for -xr
the first time.
shell# ls -althrst /var/lib|grep etcd
0 drwxr-xr-x 2 etcd etcd 6 Nov 5 14:19 etcd
Thanks to google and stackoverflow (https://askubuntu.com/questions/152001/how-can-i-get-octal-file-permissions-from-command-line):
shell# stat -c "%a %n" /var/lib/etcd/
755 /var/lib/etcd/
Much easier!
cat /tmp/dirs | xargs -n 1 stat -c '%y %n'
%x Time of last access
%y Time of last modification
%z Time of last change
sometimes you have some disconnected session that still forces some pane to be non-full size.
you can force a full resize again using: ctrl
+b
, :detach-client -a
.
Running an X application in docker I usually google for the right parameters, but sometimes they forget the X authority file. Now I use this:
#!/bin/bash
xhost +
docker run --name=foobar --privileged -it -v $PWD/../:/articlemanager --workdir /articlemanager \
-e DISPLAY \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-v $HOME/.Xauthority:/root/.Xauthority \
--net=host \
foobar:latest xeyes
If you want to do it in a more correct/secure way, I found this blog post: http://wiki.ros.org/docker/Tutorials/GUI
Also found some useful info here: https://stackoverflow.com/a/44434831/1958831 (In my above example the net=host was needed too, but it's annoying port 80 is already in use for example.)
There are some interesting snippets here that I need to extract:
https://stackoverflow.com/questions/3601515/how-to-check-if-a-variable-is-set-in-bash
usermod -a -G docker ray
if you are added to a group, you can do newgrp docker
or login again.
sudo ip addr add 192.168.2.150/24 dev enp0s3
ip addr
Available on most Linux distro's, Raspberry PI's and Arduino devices.
It's a convenient way to allow devices to find each other on a network Now I let the raspberry pi's here publish some service named "_ray" so I can easily find them:
trigen@zenbook:~> avahi-browse -d local _ray._tcp --resolve -t
+ wlp3s0 IPv6 dashboardpi _ray._tcp local
+ wlp3s0 IPv4 domoticapi _ray._tcp local
+ wlp3s0 IPv4 dashboardpi _ray._tcp local
= wlp3s0 IPv6 dashboardpi _ray._tcp local
hostname = [dashboardpi.local]
address = [fe80::90c3:716c:7d24:ccc9]
port = [80]
txt = []
= wlp3s0 IPv4 domoticapi _ray._tcp local
hostname = [domoticapi.local]
address = [192.168.115.1]
port = [8083]
txt = []
= wlp3s0 IPv4 dashboardpi _ray._tcp local
hostname = [dashboardpi.local]
address = [192.168.2.126]
port = [80]
txt = []
Then:
pi@domoticapi:~ $ cat /etc/avahi/services/ray.service
<?xml version="1.0" standalone='no'?><!--*-nxml-*-->
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
<name replace-wildcards="yes">%h</name>
<service>
<type>_ray._tcp</type>
<port>8083</port>
</service>
</service-group>
and restart the avahi-daemon
Source: https://serverfault.com/questions/556667/how-do-i-figure-out-which-drive-is-failing
for x in /sys/block/sd*
do
dev=$(basename $x)
host=$(ls -l $x | egrep -o "host[0-9]+")
target=$(ls -l $x | egrep -o "target[0-9:]*")
a=$(cat /sys/class/scsi_host/$host/unique_id)
a2=$(echo $target | egrep -o "[0-9]:[0-9]$" | sed 's/://')
serial=$(hdparm -I /dev/$dev | grep "Serial Number" | sed 's/^[ \t]*//')
echo -e "$dev \t ata$a.$a2 \t $serial"
done
in my case ata5 was throwing errors in dmesg -w
,
output:
sda ata1.00 Serial Number: S12RNEAD500626J
sdb ata2.00 Serial Number: W4Z1GZLP
sdc ata3.00 Serial Number: R6GGJN0Y
sdd ata4.00 Serial Number: WD-WMC4M0F6RXR8
^Csde ata5.00
sdf ata6.00 Serial Number: ZA19BLKY
sdg ata7.00 Serial Number: ZA19Z3D9
sdh ata8.00 Serial Number: 45D0LWNAS
caveat: this command blocks if the hdd is broken, so in that case you want to make an async version of this command. but in my case I was lucky enough that control+c already told me sde
was the bad apple.
In Vim, you can insert it with Ctrl-v
x
a
0
git diff HASH^!
diff against it's parent, sometimes reversed for me
I then simply use
git diff HASH~1 HASH
git reset --soft HEAD~1 # also creates the reference ORIG_HEAD
git commit --reuse-message=ORIG_HEAD # can be used to re-use the message
package pax-utils
lddtree $BINARY
Source:
https://askubuntu.com/questions/123798/how-to-hear-my-voice-in-speakers-with-a-mic
I'm using pavucontrol
pactl load-module module-loopback latency_msec=1
To unload loopback
pactl unload-module module-loopback
nvram set wl_regdomain="EUROPE" nvram set wl_country_code="EU" nvram commit
to enable channel 13 and 14
https://unix.stackexchange.com/questions/42856/how-can-i-convert-a-png-to-a-pdf-in-high-quality-so-its-not-blurry-or-fuzzy
img2pdf -o sample.pdf sample.jp2
lpstat -R
foo.py writing to fd 3, ./foo.py 3>&1
or beforehand:
exec 3>&2
lsof +L1
Using rsync to merge two dirs with changes in both dirs. Check if rsync won't overwrite anything:
# dry run of rsync, note no --delete
rsync --dry-run -raPv owncloud_deskmini/ owncloud/ |& tee out.log
# dry run of rsync with "u" flag that ignores updates
rsync --dry-run -rauPv owncloud_deskmini/ owncloud/ |& tee out2.log
diff out.log out2.log
If files show up in the diff those might be overwritten
coredumpctl -1 debug
/var/lib/systemd/coredump
foo="12345"
# keep foo only if it contains a valid integer, otherwise truncate
[[ $foo =~ ^-?[0-9]+$ ]] || foo=""
cat << EOF > file.yaml
Contents
EOF
The above has a flaw, it will replace for example $1 stuff, but if you put the EOF in single quotes, it will preserve it:
cat << 'EOF' > script.sh
Contents $1
EOF
This is also possible:
if [[ -z $1 ]]; then
cat <<- EOF > file.yaml
Contents
EOF # as long as EOF is indented with TAB character(s)
fi
https://www.gnu.org/software/bash/manual/bash.html#Here-Documents
trigen@xps:~/projects/jirahours[master]> date +"%Y-%m-%d %T +0200 CEST"
2021-06-01 14:04:56 +0200 CEST
root@rb-b91-s15sp2-03-21-a:~# date +"%Y%m%d%H%M%S"
20220321135439
sudo dpkg-reconfigure tzdata
to fix the timezone.
source: https://help.ubuntu.com/community/UbuntuTime
gdb -p $(pidof myprogram) < <(printf "yes\nset pagination off\nset logging on\nbt\nthread apply all bt\ninfo threads\nquit\n")
xset dpms 0 0 0 && xset s noblank && xset s off
mysqldump --skip-extended-insert --default-character-set=utf8mb4 -u root -pPASSWORD wordpress -r ./wordpress.sql
mysql --default-character-set=utf8mb4 --protocol=TCP --host 127.0.0.1 -uroot -pPASSWORD --database=wordpress < wordpress.sql
comma-separate multiple lines
$ awk -v ORS=, '{print $2}' data.txt | sed 's/,$//'
+12.0,+15.5,+9.0,+13.5
i.e.;
std::ifstream fi;
fi.open("all.csv");
std::string line;
while (getline(fi, line, '\n')) {
std::cout << "line = " << line << std::endl;
}
writing:
#include <fstream>
std::ofstream log_file(
"log_file.txt", std::ios_base::out | std::ios_base::app );
log_file << text << std::endl;
Initialize N elements at once with: vector<int> vec_out(vec_in.size(), 0);
.
Put last elem in front data[0] = data[data.size() - 1];
and erase last data.erase(data.begin() + (data.size() - 1), data.end());
Sum accumulate(count.begin(), count.end(), 0)
.
Rotate 1 2 3 -> 2 3 1 with rotate(A.begin(), A.begin() + K, A.end());
, the other way around with rbegin()
and rend()
respectively.
Find std::find(std::begin(c), std::end(c), 1001) != std::end(c);
Erase + remove idiom: c.erase(std::remove(std::begin(c), std::end(c), 1001), std::end(c));
std::vector<Foo> foos;
for (int i = 0; i<10; i++) {
foos.push_back(Foo(i));
}
std::vector<int> xs;
xs.resize(foos.size());
std::transform(foos.begin(), foos.end(), xs.begin(), [](Foo f){return f.x;});
from: https://stackoverflow.com/questions/39844106/extract-elements-from-a-vector-of-object
with ranges:
#include <iostream>
#include <range/v3/view.hpp>
struct Object {
int x;
float y;
};
int main() {
std::vector<Object> objs = {{1, 4.2f}, {42, 5.1f}, {69, 6.9f}};
std::vector<int> xs = objs | ranges::view::transform(&Object::x);
for (auto x : xs) {
std::cout << " " << x;
}
std::cout << std::endl;
for (auto y : objs | ranges::view::transform(&Object::y)) { // No new vector created.
std::cout << " " << y;
}
std::cout << std::endl;
}
#include <boost/algorithm/string/join.hpp>
#include <vector>
#include <iostream>
int main(int, char **)
{
std::vector<std::string> list;
list.push_back("Hello");
list.push_back("World!");
std::string joined = boost::algorithm::join(list, ", ");
std::cout << joined << std::endl;
}
std::string s = std::accumulate(std::next(v.begin()), v.end(),
std::to_string(v[0]), // start with first element
[](std::string a, int b) {
return a + '-' + std::to_string(b);
});
template <typename... Params>
void remove(Params &&... node) {
static_assert(std::is_constructible<Type, Params...>::value, "parameters are invalid for constructing a Type");
auto n = std::make_unique<value_type >(std::forward<Params>(node)...);
remove(static_cast<const value_type &>(*n.get()));
}
#include <sstream>
stringstream ss;
ss << "hello world" << 123 << endl;
ss.str(); // ...
ss.clear();
ss.str(""); // re-use.
std::sort (v.begin(), v.end()); // 10 10 10 20 20 20 30 30
auto low = std::lower_bound (v.begin(), v.end(), 20); // ^ ^
auto up = std::upper_bound (v.begin(), v.end(), 20); // ^
// "if the number is less.. goes down the pile"
priority_queue<int, vector<int>, std::less<int>> pq;
for (int i : {1, 6, 3})
pq.emplace(i);
pq.top(); // 6
pq.pop();
pq.top(); // 3
Exposes it's container as protected
member variable. In order to access, you have to extend priority_queue
.
Underlying data structure is a heap.
bitset<8> bs;
bs.set(0);
bs.set(2); // 00000101
Underlying container type is a binary tree. So most stuff is log n
. Note that a binary tree is also sorted, so the first element *the_set.cbegin()
is the min()
element!
unique_ptr<foo> p{nullptr};
p = make_unique<foo>();
Perfect forwarding variable template args
namespace std {
template<typename T, typename ...Args>
std::unique_ptr<T> make_unique(Args &&...args) {
return std::unique_ptr<T>(new T(std::forward<Args>(args)...));
}
}
std::numeric_limits<size_t>::max();
#include <iterator>
copy(tokens.begin(), tokens.end(), ostream_iterator<string>(cout, "\n"));
void split(string in, vector<string> *tokens_ptr, string delim = " ") {
vector<string> &tokens = *tokens_ptr;
...
split(input, &tokens, " ");
// find
string s("aa b c");
s.find(" "); // finds 2
s.find(" ", 3); // finds 4 (starting search from the "b")
// substr()
s.substr(0, 2); // start + length!
s.substr(pos, pos - previous); // typical
getline()
really nice implementation using getline()
from this stackoverflow answer.
#include <string>
#include <sstream>
#include <vector>
using namespace std;
void split(const string &s, char delim, vector<string> &elems) {
stringstream ss(s);
string item;
while (getline(ss, item, delim)) {
elems.push_back(item);
}
}
vector<string> split(const string &s, char delim) {
vector<string> elems;
split(s, delim, elems);
return elems;
}
ostream_iterator<T>
and copy()
#include <iterator>
#include <algorithm>
// output even a newline after each element
copy(c.begin(), c.end(), ostream_iterator<string>(cout, "\n"));
xxd -i favicon.ico > favicon.ico.h
template <typename T>
std::string join_keys(std::vector<T> &idsVec) {
std::stringstream ids;
// remove dupes
std::sort(idsVec.begin(), idsVec.end());
idsVec.erase(std::unique(idsVec.begin(), idsVec.end() ), idsVec.end());
// join ids with ","
std::copy(idsVec.begin(), idsVec.end(), std::ostream_iterator<T>(ids, ","));
std::string ids_str = ids.str();
// remove the trailing ", "
ids_str.erase(ids_str.empty() ? 0 : ids_str.length() - 1);
return ids_str;
};
c++17 will support it, in g++ version 7.0 it can be enabled with --std=c++1z
#include <optional>
...
std::optional<std::string> empty;
std::optional<std::string> filled{"hello"};
if (filled) {
std::cout << "filled = " << *filled << std::endl;
}
std::cout << "empty = " << empty.value_or("nothing") << std::endl;
c++14 flag --std=c++1y
in g++ 4.9 supports std::experimental::optional
#include <experimental/optional>
...
std::experimental::optional<int> empty;
Note there is also make_optional
.
using namespace std;
ifstream ifile("webroot/"s + folder + "/" + file, ios::binary);
string s( (istreambuf_iterator<char>(ifile)),
(istreambuf_iterator<char>()) );
cout << "read = " << s.size() << " & " << s.length() << endl;
const char * long_test = R"( ... )";
const char * long_test = R"XXX( ... )XXX";
Passing optional reference:
#include <iostream>
#include <experimental/optional>
#include <functional>
namespace std {
using std::experimental::optional;
}
void foo(std::optional<std::reference_wrapper<bool>> flag = {}) {
if (flag) {
flag->get() = true;
}
}
int main() {
bool flag = false;
foo({flag});
std::cout << "flag is now: " << std::boolalpha << flag << std::endl;
}
/* sleeping */
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
/* timing */
auto last_received = std::chrono::high_resolution_clock::now();
...
auto current_time = std::chrono::high_resolution_clock::now();
std::chrono::duration<double, std::milli> idle = current_time - last_received;
if (idle.count() > 500.) {
/* literals */
using namespace std::literals;
std::this_thread::sleep_for(0.5s);
}
#include <algorithm> // copy
#include <iterator> // back_inserter
std::copy(from_vector.begin(), from_vector.end(), std::back_inserter(to_vector));
void OpenStackStrategy::apply(CustomizationFileEntryPairList &customizations)
{
size_t index = customizations.size();
for (const auto &customization : boost::adaptors::reverse(customizations)) {
const auto &file = customization.first;
const auto &entry = customization.second;
index--;
std::cout.precision(17);
std::cout << std::fixed << some_double_variable << std::endl;
std::cout << std::boolalpha << some_boolean_variable << std::endl;
std::sort(ignore_domains.begin(), ignore_domains.end());
std::sort(search_domains.begin(), search_domains.end());
std::vector<std::string> result;
std::set_difference(
search_domains.begin(), search_domains.end(),
ignore_domains.begin(), ignore_domains.end(),
std::back_inserter( result )
);
auto now = std::chrono::system_clock::now();
auto epoch = std::chrono::system_clock::to_time_t( now );