In shell scripting I prefer the Kornshell. A while ago I experimented with "oh my zsh", but I switched back to ksh. Their auto completion for program commands is really unsurpassed (tab completion on program parameters for grep for example). The auto-incorrect however, is quite annoying . There is also a git plugin that visualized the active git branch in the $PS1 prompt. I liked these features and I want to add them to ksh.

Apparently I like old things

I fixed a few things for ksh in my .kshrc at bitbucket.

Separated history amongst the different ksh shells.

My problem with the default behaviour: by default history is globally shared amongst all shells. I tend to work in a screen and do different stuff in each buffer. So it's annoying if stuff from one buffer magically appears in the other buffer.

So what I do is I make sure there is one history file ~/.ksh_history which contains all history from all shells. When starting a new shell I copy this file into a history file specific for that ("sub")shell, i.e. ~/.ksh_history_files/<shellpid>. Each new shell does this.

When a shell is started all history files from ~/.ksh_history_files/* are merged back into ~/.ksh_history. And the ones that are no longer in use (shells have exited) are removed. This is done with a simple lsof call.

Commands are processed through a simply ksh function that makes sure all history lines are unique, without changing the order.

Some funny caveats were:

• A history file should start with the character sequence \x81\01.
• Each command in history file should end with a \x00 character.

A prompt that embeds current git branch

If you are inside a git clone).
Maybe the oh-my-zsh git integration is more advanced, no idea. Luckily this visualization in $PS1 is very fast.

trigen@Firefly21:/projects> cd smashbattle
trigen@Firefly21:/projects/smashbattle[network_multiplayer]> git branch
  master
* network_multiplayer
trigen@Firefly21:/projects/smashbattle[network_multiplayer]> 

Fix "cd" (with no params) to be the equivalent of "cd ~" again

Fixed with a simple alias around cd provided by /usr/share/ksh/functions/dirs. For dirs usage check here.

trigen@Firefly21:/usr/local/src/wxWidgets-3.0.0> cd <<<< this didn't work on Ubuntu 13.10 anyways.
trigen@Firefly21:/home/trigen> 

Integrated my "launcher tool" I wrote for windows a long time ago.

The browser is assumed to be "chromium-browser" (sudo apt-get install chromium-browser).

The result for gi dan flavin is opened in a new chromium tab.

• i <url> - open url in browser
• g <search terms> - search with google.com
• gi <search terms> - search with google images
• gv <search terms> - search with google videos
• gnl <search terms> - search with google.nl
• gs <search term> - search google scholar
• w <search term> - search wikipedia
• wa <search terms> - search wolfram alpha
• yv <search terms> - search youtube
• y <search terms> - search yahoo
• yi <search terms> - search yahoo images
• yv <search terms> - search yahoo videos
• imdb <search terms> - search imdb
• h <search term> - search hyperdictionary
• v <search term> - search vandale (dutch dictionary)

Usage example: Open run menu (ALT+F2). Then type gi dan flavin to get the example result from screenshot.

I had to abandon my pure-ksh-functions approach to make the launcher commands available everywhere. So you have call install_launcher as root to install the shortcuts as scripts in /usr/local/bin/. Ubuntu does not respect shell functions in run unfortunately.

As a free alternative for Charles proxy and/or Fiddler I developed wxHttpProxy in one (long) night! Unfortunately I had to spend the next day making it stable, and support https traffic as well

The mentioned alternatives are really great by the way, and have quite a few additional features, I just needed a simple http proxy that I could use in metalogmon, that's why I developed my own. It is opensource, find it on bitbucket.

The idea is to integrate this http proxy into metalogmon, so that you would be able to process your output through javascript. The reason that request/response are both first class citizens is that where I currently develop software I plan to use this proxy between all subsystems communication, and here multiple services communicate with eachother using REST calls.

I want to visualize the call sequences/hierarchy, like [A -> B -> C], [A <- B <- C]. And not group them together like: [A <> B], [B <> C].

Install binary (64 bit and ubuntu/debian based system only)

Install my key

wget -O - http://cppse.nl/apt/keyFile | sudo apt-key add -

Add my repo to apt's sources.list.

sudo sh -c "echo deb http://cppse.nl/apt/dists/stable/main/binary / >> /etc/apt/sources.list"

Install the package:

sudo aptitude update
sudo aptitude install wxhttpproxy

Usage

wxHttpProxy

Tip of the day, add a shortcut for starting the proxy (like WINKEY+2 or something)

Available shortcuts

• alt+i - Toggle intercept mode (capture/display request/response, or be silent)
• alt+c - Clear buffer(s)

Catch library

With Catch I developed a few helper classes in a test-driven way, it's a lightweight single-header c++ library for creating unit tests. Really liked working with it and look forward to using it in the future some more.

The apt repository

The shiny new apt repository was also really easy to setup, once you have made your packages, simply follow this wiki.

Other platforms

Should you wish to compile it for windows, it is wxWidgets 3.0.0 based and should already be cross-platform, just didn't test it yet. Use Dialogblocks to compile it (and also have it compile wxWidgets for you (as that's just convenient )).

Implemented this feature I saw in XChat

Yes, having switched to Linux and using XChat, I now first realize how slow mIRC is (connecting to my bouncer, joining all kinds of channels, replaying scrollback, etc.-- mirc takes a few seconds, where XChat is "instant"), but I don't care. It's still my client of choice for Windows. On Tweakers IRC network, it is custom for a lot of people to indicate their away status with /nick <nickname>|afk and such. That makes sense, but XChat does something "smart", it periodically /who's all channels to know whom are away so it can color those in light grey in the nicklist. Cool feature I thought! So I "extended" my nicklisting coloring mirc script, by adding aways.ini. It's no longer necessary to keep changing your nickname with an away status (on freenode it isn't even allowed in some channels ).

Away nicks in light-grey in XChat

The same feature in mIRC

It's a pity that such an inefficient implementation is required ("pull"ing /who periodically will result in a lot of extra network data), but the IRC protocol does not currently provide something that "push"es away info more intelligently. You can download the script here, note that aways.ini depends on nicklist.ini, also downloadable there. (nicklist.ini still works without aways.ini, it will then use light grey at random instead.)

Away nicks in light-grey


Away nicks in light-grey, other nicks colored.

These are the ones I keep forgetting about:

• Enabling line numbers: Tools, Options, Text Editor, All Languages, Show Line Numbers (source: http://stackoverflow.com/questions/5905687/can-i-see-line-numbers-in-the-vs2010-editor)
• Toggle whitespace/tabs: CTRL+R, CTRL+W. (source: http://stackoverflow.com/questions/4065815/how-to-turn-off-showing-whitespace-characters-in-visual-studio-ide)
• Navigate cursor to previous+next position: CTRL+- and forward CTRL+SHIFT+-.

These are a few you can't do without:

• Step into function: F12 (CTRL+- to return cursor to previous position)

If you can afford it you can't do without Visual Assist X either.

Click the play button to start stream. (Will only work if I have the script running on my raspberry pi though. )

raspivid + ffmpeg

# test chaining raspivid and ffmpeg
raspivid -t 5000 -w 960 -h 540 -fps 25 -b 500000 -vf -o - | ffmpeg -i - -vcodec copy -an -r 25 -f flv test.flv

# stream to tcp endpoint
raspivid -n -t 0 -w 1920 -h 1080 -fps 25 -b 2000000 -o - | ffmpeg -i - -vcodec copy -an -f flv -metadata streamName=video2 tcp://cppse.nl:6666

Port 6666 is the same with the sample that is delivered with crtmpserver (the flvplayback.lua sample).

Turn off pi camera led

#!/usr/bin/env python
import time
import RPi.GPIO as GPIO

# Use GPIO numbering
GPIO.setmode(GPIO.BCM)

# Set GPIO for camera LED
CAMLED = 5

# Set GPIO to output
GPIO.setup(CAMLED, GPIO.OUT, initial=False)

# Five iterations with half a second
# between on and off
#for i in range(5):
#  GPIO.output(CAMLED,True)  # On
#  time.sleep(0.5)
#  GPIO.output(CAMLED,False) # Off
#  time.sleep(0.5)

GPIO.output(CAMLED,False) # Off

Think I got this script from here.

Live video streaming

I was interested in specifically the generation of H264 videos directly from code (without audio). This was something I used to do by generating separate .BMP files that I then convert with mencoder. This costs quite a lot of hdd space and was performance-wise not optimal. Of course I already figured that out, but figuring out how to encode directly to video was something on my To Do list.

Believe it or not, I actually had to search a while for libraries and SDK's, before concluding that ffmpeg was the best option. I've had a bad experience with ffmpeg 5 years ago, so I was probably ignoring ffmpeg in the google search results. You can link to ffmpeg's libraries and use them in your own program, they also provide code samples. This made it all pretty easy to setup. So when I completed generating H264 videos from code I wondered how difficult it would be to directly stream video with ffmpeg.

One cool application would be to use it in games, f.i. make a small livestream for actual games being played on the server.

Ffmpeg supports a few protocols, I chose an RTMP server for my setup, which is Flash only. If that works it'll probably work for other protocols as well. The best alternative for HTML5 is probably Apple's HLS Streaming, which actually is pretty awesome, as a side-note from source:

If multiple qualities of a stream are available, the player will continuously monitor the current bandwidth and pick the next fragment from the highest quality it can load. This is called Adaptive Streaming:

For the videoplayer I chose JWplayer version 6.

Setting up an rtmp server with Jwplayer

The servers I tested: rtmplite on linux, crtmpserver on linux and Red5 Media Server on windows. The linux server I used was a Suse (SLED11) micro EC2 instance on Amazon AWS. But these should all work on linux and windows.

RTMP is flash-only, traffics over port 1935.

Some notes on using rtmplite with JWplayer

Eventually I chose not to use rtmplite, but crtmpserver instead, as you have more output there as to what's going on in the server. rtmplite output is less useful to me, by default it outputs nothing, with verbose you get messages at packet level. rtmplite used quite a bit more of my CPU compared to crtmpserver (not very accurate due to EC2, but crtmpserver at 0.3% CPU versus > 10% CPU for rtmplite) and it does not understand aspect ratio other than 4:3, so 16:9 will be "squished" into 4:3. It's not my intent to be negative about rtmplite, if you're a python guy It's probably very easy to tweak and I must say it works out-of-the-box with a simple python rtmp.py, ready for video at rtmp://servername/app/video.

With JWplayer you can view the stream with:

jwplayer("myElement").setup({
    file:       "rtmp://localhost/appflv:video",
    autostart:  true,
    width:      480,
    height:     320
});

If you attempt to use that config, please note that file URL! It took me an hour to get that correct, in all JWPlayer samples you see rtmp://server/app/flv:video.flv or rtmp://server/app/mp4:video.mp4. The prefix flv: or mp4: being just for JWplayer so that it knows what encoding to expect. As the .mp4 or .flv extension in URL's is not mandatory. Eventually JWplayer will strip away flv:. An rtmp url like rtmp://server/app/video consists of an application called app and a video called video. Stripping away the prefix from rtmp://cppse.nl/app/flv:video will provide app/ for the application which looks like app but isn't quite the same.

So... crtmpserver then!

After compiling you fire it up with default settings with: ./crtmpserver/crtmpserver crtmpserver/crtmpserver.lua. You should see something similar to the following output.

ip-10-224-83-43:/usr/local/src/crtmpserver/builders/cmake # ./crtmpserver/crtmpserver ./crtmpserver/crtmpserver.lua
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:203 Initialize I/O handlers manager: epoll
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:206 Configure modules
/usr/local/src/crtmpserver/sources/thelib/src/configuration/module.cpp:84 Module /usr/local/src/crtmpserver/builders/cmake/applications/appselector/libappselector.so loaded
/usr/local/src/crtmpserver/sources/thelib/src/configuration/module.cpp:84 Module /usr/local/src/crtmpserver/builders/cmake/applications/flvplayback/libflvplayback.so loaded
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:212 Plug in the default protocol factory
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:219 Configure factories
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:225 Configure acceptors
/usr/local/src/crtmpserver/sources/thelib/src/netio/epoll/iohandlermanager.cpp:100 Handlers count changed: 0->1 IOHT_ACCEPTOR
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:231 Configure instances
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:237 Start I/O handlers manager: epoll
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:240 Configure applications
/usr/local/src/crtmpserver/sources/thelib/src/configuration/module.cpp:177 Application appselector instantiated
/usr/local/src/crtmpserver/sources/thelib/src/configuration/module.cpp:177 Application flvplayback instantiated
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:246 Install the quit signal
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:257
+-----------------------------------------------------------------------------+
|                                                                     Services|
+---+---------------+-----+-------------------------+-------------------------+
| c |      ip       | port|   protocol stack name   |     application name    |
+---+---------------+-----+-------------------------+-------------------------+
|tcp|        0.0.0.0| 1935|              inboundRtmp|              appselector|
+---+---------------+-----+-------------------------+-------------------------+
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:258 GO! GO! GO! (19664)

With the default configuration anyone can stream a video to the RTMP server, as inbound and outbound are over port 1935. I tried separating that with the configs, but I couldn't find any configuration setting to do this, so I hacked something in the source to only allow inbound RTMP for 127.0.0.1 (see patch.txt). With crtmpserver you don't have the URL problem in JWPlayer.

jwplayer("myElement").setup({
    file:       "rtmp://server/flvplayback/flv:video",
    autostart:  true,
    width:      480,
    height:     320
});

Sending video to the server with "sendstream"

The proof-of-concept tool I set out to make is now a useful test tool you can use to stream a test video with optional audio at a given resolution to a given streaming server. I have compiled it for windows and linux 32 bit (opensuse 12) and also have that in a working chroot for download (~28 mb zipped).

usage: sendstream.exe < RTMP URL > [ < width > < height > < sound > < max_fps >]
 params: width default = 320, height default = 240 pixels, sound default = 1, max_fps default = 25.
 i.e: sendstream.exe "rtmp://localhost/app/video" 800 600

Example output for sendstream.exe rtmp://cppse.nl/flvplayback/video 1920 1080 1:

[libmp3lame @ 058a6320] Channel layout not specified
Output #0, flv, to 'rtmp://cppse.nl/flvplayback/video':
    Stream #0:0: Video: flv1, yuv420p, 1920x1080, q=2-31, 100 kb/s, 90k tbn, 20
tbc
    Stream #0:1: Audio: mp3, 44100 Hz, 2 channels, s16p, 24 kb/s
HandShake: client signature does not match!
writing frame 0 at 0.044793... +V
writing frame 1 at 0.112187... +A+A+A[flv @ 058a2b60] Encoder did not produce pr
oper pts, making some up.
+A+V
writing frame 2 at 0.324518... +A+A+A+V
writing frame 3 at 0.813168... +A+A+A+A+A+A+A+A+A+V
writing frame 4 at 1.03129... +A+A+A+A+A+A+A+A+A+A+A+A+A+A+A+A+A+A+A+V
writing frame 5 at 1.16484... +A+A+A+A+A+A+A+A+A+V
writing frame 6 at 1.23609... +A+A+A+A+A+V
writing frame 7 at 1.28777... +A+A+A+V
writing frame 8 at 1.33955... +A+A+V
writing frame 9 at 1.38883... +A+A+V
writing frame 10 at 1.43901... +A+V
writing frame 11 at 1.48954... +A+A+V
writing frame 12 at 1.53965... +A+A+V
writing frame 13 at 1.58858... +A+A+V
writing frame 14 at 1.6456... +A+A+V
writing frame 15 at 1.69497... +A+A+V
writing frame 16 at 1.74456... +A+A+V

Audio and video are combined with interweaving and if rendering of video frames is relatively slow compared to audio it may require more audio frames in between and vica versa. In the output +A is a written audio and +V a written video frame.

I use SFML for generating the graphics and for audio something similar to the example code by ffmpeg. The interweaving part was the most difficult but important thing to get right. This is the easiest version of the code I could come up with that works properly. Something I didn't know in the beginning was that you don't need to keep your framerate of sending frames to the server consistent at a given frames per second-rate.

sfml_init(&pixels, &pixelSize);

auto timer = TimerFactory::factory(TimerFactory::Type::WindowsHRTimerImpl);
timer->start();

frame->pts = timer->end();

int64_t currentframe = 0;
while (true) {

    sfml_generate_frame();

    std::cout << "writing frame " << currentframe++ << " at " << (timer->end() / 1000.0) << "... ";

    // Make sure to write enough audio frames until the audio timer lines up
    while (true) {
        double audio_pts = (audio_st) ? (double)audio_st->pts.val * audio_st->time_base.num / audio_st->time_base.den : 0.0;
        double video_pts = (video_st) ? (double)video_st->pts.val * video_st->time_base.num / video_st->time_base.den : 0.0;

        if (audio_pts >= video_pts)
            break;

        std::cout << "+A";
        write_audio_frame(oc, audio_st);
    }

    // Write video frame
    write_video_frame(oc, video_st);
    std::cout << "+V" << std::endl;

    // Set elapsed time as frame time
    frame->pts = timer->end();
}

One final side-note if you are going to use sendstream on linux with Xvfb (X virtual framebuffer). On SUSE it was a pain to get it working, I ended up installing it on an openSuse VMWare VM where it did work and then made a minimal chroot for it, one that includes Xvfb and sendstream and a run.sh script (you might want to edit that). Not all screen resolutions you can think off that work fine with RTMP server will work, i.e.: I wanted a 980 x 100 resolution video, that didn't work for Xvfb, it does work if you create a larger screen with a more common resolution like 1280 x 768. Also 980 x 100 didn't work for SFML, 1000 x 100 did, probably the OpenGL underneath complaining or something. AFAIK these problems do not exist without Xvfb.

The result

Sendstream runs on my webserver capped at 2 FPS and without audio, so that it doesn't hurt the CPU too much.

Download: sendstream-1.0.zip for windows. 11.2MB

Download: sendstream-1.0-chroot.zip for linux (chroot, use: "chroot /path/to/sendstream /run.sh"). 27.2MB

a930805b8867232755da195d34587476 *sendstream-1.0-chroot.zip
6ed8cf5102dbe10c5f4722c80d38cbbd *sendstream-1.0.zip

Het thema van de Dutch PHP Conference 2013 was Respecteer het eco-systeem.

Dag één begon voor mij (en mijn collega's) 7 juni en Ade Oshineye opende met het pleiten voor User Journeys, de flows over meerdere devices die gebruikers volgen zoals openen van een nieuwsbrief op een telefoon en naar doorklikken naar product om deze uiteindelijk te kopen op een werkstation.

Denk ook na over dat een website geopend in een LinkedIn app nog kleiner getoond wordt (aangezien het in een webview geladen wordt, soort een frame). Hoe ziet je site er dan nog uit? Hoe vaak komt het niet voor dat je vanuit een nieuwsbrief door wilt klikken naar een product om vervolgens geconfronteerd te worden met een inlogscherm? Kan dat niet gebruiksvriendelijker?

Unbreakable Domain Models van Mathias Verraes bevatte tips die mij doen denken aan OO in C++: goed gebruik maken van types. I.p.v. een string voor een e-mail echt een Email class maken die eventueel ook validatie regelt. Wat ik interessant vind om te zien is dat de spreker in zijn sheets gebruik maakt van exceptions in de constructor. Waar ik persoonlijk ook groot fan van ben maar veel mensen in mijn omgeving niet . Naar mijn mening moet je een type niet half kunnen aanmaken (constructen) want dan is het geen volledig type, maak dan een liever een ander type die die state representeert. Zijn slides over "Encapsulate operations" tonen gebruik van een CustomerSpecification interface en het maken van classes zo dicht mogelijk bij het domein.

Surviving a Prime Time TV Commercial - David Zuelke. Zeer interessante manier van vertellen met veel tips en info, triggert mij te kijken naar e.e.a., zoals genoemde supervisord, transactional e-mail of elastic search. Voor hun use-case was het zinvol om zo'n beetje alles in Elastic Search op te slaan.

PHP, Under The Hood - Anthony Ferrara. Ook goede spreker en presentatie was humorvol. Hij heeft een PHP compiler gemaakt in PHP en wist leuke dingen te vertellen over opcodes en liet bijbehorende code hier en daar zien. Helaas talk maar voor de helft gezien. Dat vm_execute.h gegenereerd wordt tijdens compileproces door PHP vind ik niet zo schokkend, dat je heel g++ compiled met g++ vind ik pas schokkend!

Mikado!

Uncon: The Mikado method - Pascal de Vink (slides here). Hele korte talk, maar wel interessant. Een methode omtrent een leuk idee, met als onderliggende toon misschien wel het tegenovergestelde van "Respecteer het eco-systeem":

• Formuleer een doel, schrijf deze op papier.
• Naief implementeren van je doel in de code, 'fuck' de details of de dingen die je stuk maakt. Gewoon coden hoe het moet worden.
• Als je dit foutloos hebt gedaan en alles werkt nog: klaar. Maaarrr waarschijnlijk heb je wel een aantal dingen stuk gemaakt, heb je een lijstje compile errors of heb je in je hoofd een aantal TODO's of FIXME's
• Je schrijft deze als dependencies onder je doel (alles met pen en papier). Dit zijn dan de prerequisites voor je doel in een boomstructuur.
• Je fixed ze als het simpel is of shelved je changes en gaat één voor één die prerequisites fixen en comitten.
• Je streept alles van je blaadje af en werkt zo gericht naar je doel toe.

Idee achter deze methode is dat je wellicht code weg moet gooien of shelven, maar daar tegenover staat dat je snel je prerequisites inzichtelijk hebt. Deze kun je ook aan een andere developer geven als een duidelijkere roadmap voor implementatie van dit doel.

Dependency Injection Smells - Matthias Noback. Volgens mij vond ik dit een saaie talk, weet het niet meer zeker. Maar wel veel herkenbare issues qua design en soort van eigenaardigheden in PHP. Zo is in PHP public function foo(Bar $baz) alleen maar afdwingen dat $baz de interface Bar implementeert. Maar is $baz gewoon de implementatie, en als je niet uitkijkt programmeer je tegen de implementatie aan i.p.v. de interface. (Gelukkig is phpstorm slim genoeg om dit snel herkenbaar te maken)

Getting your toolbox together - Daan van Renterghem. Was een leuke talk, ging over Vagrant. Persoonlijk nooit een hobby van me geweest het configureren van dit soort dingen, maar hij liet Vagrant zien en vertelde over Chef en dat was zeker wel interessant.

De volgende dag...

Growth Hacking for Humans - Eamon Leonard. (video) Zeer interessante keynote. Een man die behoorlijk ondernemend is en een aantal fasen heeft doorlopen die hij wilde delen met de rest. Benadrukte de belangrijke rol van communities.

Scenario Driven API Design - Ivo Jansch. Sta je op het punt een (REST) API te ontwikkelen, neem dan deze talk even door. Leuke anecdote, spreker: "Wie heeft er wel eens wat gehoord over Scenario driven design, buiten deze talk om dan natuurlijk?". Eén of twee mensen steken hun vinger op. "Oh, dat is grappig want ik heb die term dus zelf verzonnen.".

Introduction to Django - Travis Swicegood. Was eerlijk gezegd best nieuwsgierig naar dit framework. Omdat ik binnenkort ook eens wat met Python wil gaan doen. Wel veel goede dingen over Django gehoord maar de spreker heeft niet echt goed de kracht van het framework over weten te brengen naar mij. Bleef overigens ook allemaal erg basic en na elke slide kwam een slide met daarop: "Are there any questions?".

Emergent Design with phpspec - Marcello Duarte. Deze meneer was er erg handig mee, hij is dan ook de man achter het project. Het is denk ik PHPUnit done right, maar dan niet zoals het ellendige SVN "CSV done right" zou moeten zijn. Het is iets breder, je werkt meer vanuit de specificaties (die geformuleerd zijn als tests). phpspec is gefocust om zo zinvol mogelijke foutmeldingen/feedback te geven en slimmer te zijn. Dat laatste wil zeggen dat het gespecte methoden of classes voor je kan aanmaken die nog niet bestaan. Het ondersteund je beter in Test of Behaviour driven development.

Measuring and Logging Everything in Real Time - Bastian Hofmann. Veel zinvolle tips m.b.t. logging. Veel voorkomende problemen is dat je meerdere servers en dus logs hebt, welke log entries overal horen nou bij elkaar. PHP heeft leuke apache_note. Andere trefwoorden: graylog2, elastic search, AMQP, logstash, graphite, statsd, boomerang.js, X-trace-id, monolog, error_log, set_error_handler, set_exception_handler. Zat een heel verhaal omheen, wat ik hier niet zal reproduceren.

Conclusie

Worse Is Better, for Better or for Worse - Kevlin Henney. Zeer boeiende spreker, kan ik echt lang naar luisteren zulke talks. Alan Kay meerdere keren geciteerd gedurende deze conference, zo ook in deze talk, zie ook phpspec slides, slide 40. Het messaging aspect tussen de objecten is onderbelicht geweest, zie slides 13, 14, 15 (erg leuk). Ik vind het "jammer" dat ik geen ervaring heb met COM, dus zal eens een keer uit nieuwsgierheid naar kijken. Hij ging in deze talk in op een uitspraak "Worse is better":

Over two decades ago, Richard P Gabriel proposed the thesis of "Worse Is Better" to explain why some things that are designed to be pure and perfect are eclipsed by solutions that are seemingly limited and incomplete. This is not simply the observation that things that should be better are not, but that some solutions that were not designed to be the best were nonetheless effective and were the better option. We find many examples of this in software development, some more provocative and surprising than others. In this talk we revisit the original premise and question in the context of software architecture.

Wat is het antwoord op deze vraag? Als Smalltalk zoveel "mooier" is, waarom programmeren we nu inmiddels niet allemaal in Smalltalk? Waarom werd C++ zo populair? Het antwoord is dat C++ het ecosysteem respecteerde. In de tijd van Smalltalk waren thuis computers nog zo snel niet en smalltalk was een stuk zwaarder, daarbij draaide het afgezonderd in een virtual machine in zijn soort van eigen ideale universum. Hierbuiten smalltalk is echter een heel eco-systeem wat in dit geval een "trage" computer is met een printer, muis, toetsenbord, etc. De spreker vertelde over hoe practisch hij Turbo C++ compiler vanaf een floppy op zijn computer zette en daarmee gelijk kon programmeren en echt dingen voor elkaar kon krijgen en dat de geschreven programma's snel waren. Dit is zijn waarheid, maar ik denk dat de kern van het ecosysteem respecteren wel een goed punt is, en voor mij zeer herkenbaar.

Om terug te komen op de keynote van dag één, waarom is het web zoals we dat nu kennen, met HTTP + HTML zo succesvol geworden? Er waren ook daar (geavanceerdere?) protocollen of in iedergeval alternatieven, zoals gopher. De rede is natuurlijk dezelfde, HTML "embraces the ecosysteem", het groeit mee met de ontwikkelingen, integreert en past zich aan. Denk aan Java applets, Flash, talloze file formats, VRML, WebGl maar ook juist integreert met andere protocollen zoals FTP--en ook al weet ik niet--vast ook gopher!

Wat ik handig vind is om snel een plaatje te kunnen delen, op IRC bijvoorbeeld. Naar tweakers.net gaan, {inloggen, }naar fotoalbum, file upload knop, naar juiste map bladeren, plaatje selecteren en dan uploaden... is voor mij dan niet snel genoeg. Vooral als ik het plaatje voor me heb staan, negen van de tien keer in Windows Explorer®™.

Ik wil gewoon de files selecteren, een sneltoets indrukken en aangeven in welke map ze moeten. Daarna wil ik de link hebben naar het plaatje en klaar. Geen frustratie. :)

Voorbeeld van upload naar tweakers.net fotoalbum

Leuke bijkomstigheid is dat je gelijk een heleboel files in één keer kunt uploaden op deze manier.

Toelichting

sanitizer.exe moet gestart zijn en luistert (standaard) o.a. op CTRL + ALT + 6. Zodra die hotkey ingedrukt wordt zal sanitizer (1) de geselecteerde files uit het actieve explorer venster lezen en wegschrijven naar het bestand selected_files.txt en (2) het php script explorer_call.php aanroepen die met dit bestand kan doen wat je wilt, in dit geval de files uploaden met curl.

Uploaden naar tweakers was het doel, maar omdat ik het überhaupt handig vind om ook andere dingen te kunnen doen met geselecteerde files in explorer heb ik het flexibel gehouden met een script. Ook vind ik het handig om allerlei dingen onder ctrl+alt+{nummer} te kunnen stoppen: zoals text in het clipboard of system calls uitvoeren. Dus ook dat heb ik wat ruimer opgezet en daar is sanitizer uitgekomen.

Sanitizer configgen voor Tweakers.net image uploads

Je moet dus wel 1337 genoeg zijn om het fotoalbum unlocked te hebben binnen tweakers, anders zul je de optie onder je profiel niet hebben. :7 Als je mij genoeg vertrouwd kun je de installer van sanitizer downloaden en uitvoeren. De installer maakt ook een example_tweakers/ directory aan met daarin een explorer_call.php, die kun je bewerken en over explorer_call.php heenplakken die een directory hoger staat. Out of de box werkt deze dus nog niet.

Ten eerste zitten in explorer_call.php ook nog wat andere probeersels van mij die je kunt negeren (opties 'c' en 's').

Voorbeeld output:

Listing of files:

 - C:\Program Files (x86)\sanitizer\example_scripts\explorer_call.php.tweakers


Please select from the following list
 - s = move to unsorted directory with extensions as subdir
 - p = upload to tweakers fotoalbum public folder
 - r = upload to tweakers fotoalbum private folder
 - c = create command prompt here
 (other) = exit


You choose (press RETURN):

Ten tweede ga ik even uit van de standaard situatie dat je een private en een public folder hebt in je fotoalbum. In mijn geval is 'r' -> upload naar m'n album 'private' en 'p' naar 'public'. Je kunt het script makkelijk uitbreiden met een extra curl request om de albums eerst dynamisch op te halen. Vergeet het dan niet in de reacties te delen ;)

Als je dit huidige script werkend wilt maken voor jezelf moet je momenteel:

1. Een TnetID (sessie id) in de sourcecode op regel 29 zetten van een actieve sessie naar tweakers.net (je zou een extra sessie kunnen maken speciaal voor dit script). Dat kun je doen door in te loggen en met Firebug je cookie uit te lezen:

2. Je fotoalbum id's in de source zetten op regels 32 en 35.

Deze staan in de URL's van de albums, in het voorbeeld 1001:

3. sanitizer.exe moet wel gestart zijn natuurlijk

Troubleshoot

• Image upload werkt niet op geselecteerde files op je desktop. Open dan eerst explorer en ga via dat ding naar de desktop..
• Lees ook de troubleshoot van sanitizer. Huidige PHP script is namelijk niet geheel fout ongevoelig opgezet. :X

Hook shortcut keys Ctrl+Alt+{0,1,2,3,4,5,6,7,8,9}

sanitizer.exe is a program that reads a config, and then registers the hotkeys Ctrl+Alt+{number}, where {number} is {0, 1, ..., 9}.

Example contents for config.ini:

[1]
type=command
value=putty -load SomeSession

[4]
type=clipbard
value=MyMysqlPasswd

[6]
type=explorer

...

(Only [0] through [9] are valid entries in the config.)

As shown in the example, available action types are:

The explorer action

This action type is special in the config: the value key in the config.ini for it is not used, so you don't need to define it.

Example use: If you assign it under hotkey Ctrl+Alt+6 as I did in the example, you can use this hotkey whenever a (Windows file) explorer is active/has focus (explorer.exe). If you trigger the hotkey it fetches the selected files and writes them to selected_files.txt in the program installation directory. As long as this file exists, the hotkey won't trigger again! It will then immediately call the explorer_call.php PHP-script, also in the installation directory, that can be defined to do whatever you want on this list of files. As long as it removes the selected_files.txt afterwards.

Contents of the default explorer_call.php script:

<?php
$files = explode("\n", trim(file_get_contents('selected_files.txt')));
unlink('selected_files.txt');

print_r($files);

system("pause");
?>

Small troubleshoot

If you crash the script, and selected_files.txt is not deleted, your shortcut key won't appear to work anymore, as the existance of this file also serves as a lock. Only when the file was processed and deleted can the explorer hotkey fire and launch the PHP script again. So when your hotkey appears non-responsive, chances are you need to delete selected_files.txt, or make the script more advanced .

Download

Download setup file: sanitizer-1.0-setup.exe.

The executable is compiled with VS2012 with platform target "Visual Studio 2012 - Windows XP (v110_xp)". VS2012 x86 redistributable is included in the setup. After install make sure to edit config.ini first, and realize that there is no GUI, the executable will run in the background. If you want to restart it use the restart_sanitizer.bat. If you want to kill it, use taskmgr or taskkill /IM sanitizer.exe.

Verify

If you start it with the sample config: Ctrl+Alt+4 should yield SomeOtherPassword in your clipboard; Ctrl+Alt+6 should print selected files in Windows explorer (do this from within explorer); Ctrl+Alt+1 should start putty.exe if you have that installed (and available in your PATH).

Windows 8 (Vista?) Disable UAC

If your system complains about not being able to write to selected_files.txt, just make it run as Administrator:

Explorer call use case: Tweakers.net foto uploader script

Update 10 jun 2013:

It's included in the installer in the example directory after install, an explorer_call.php script that makes it possible to upload images to your Tnet foto-album. Wrote a blog-post about it in Dutch here: http://cppse.tweakblogs.net/blog/9047/tweakers-fotoalbum-images-uploader.html.

Update 20 oct 2013:

Made a few changes to the tweakers script again, another Tweakers user wrote a blog about this: http://perkouw.tweakblogs.net/blog/9474/t-punt-net-fotoalbum-script!.html.

• Now the only thing you need to edit is the TnetID (session id).
• It will fetch the available foto albums itself.
• I've also improved the check whether a file upload succeeded, it fetches the last known image from the fotoalbum, before and after the upload, if they match, the upload failed.
• A summary of the uploads is now opened after the last image upload, instead of opening an url every upload ;)

(I'll update the installer later to include this new explorer_call.php)

Background

Yet another C++ benchmark library I guess, but I couldn't find any that fulfilled my needs properly The requirements: output to text into a format easily parsable (for generating grahps afterwards). Low coupling, non intrusive, performant, provide basic statistics, ... .

So this one is easy to use, at first I implemented the timer using boost::posix_time::microsec_clock, only to discover that the resolution was not as good on windows. [Edit 15-02-2015: I now think the individual measures may be off a bit, but overall it does seem to average out to correct timings. I now added a boost::chrono based timer, which seems to offer good results on all platforms.] I don't understand why exactly, so I searched for a better timer (for windows) and found an example for an HRTimer [Edit 15-02-2015, forgot the link, but I found this]. So now you can choose Timer implementations ^_^

Available Timer implementations are:

• BoostTimerImpl - microseconds resolution on *nix systems, at most milliseconds on windows.
• BoostChronoTimerImpl - microseconds resolution on *nix systems and windows.
• WindowsHRTimerImpl - microseconds resolution on windows, windows specific.

Available Measure classes are:

• MeasureCounts - group individual measures per time interval.
• MeasureInterval - individual measures.

With both classes you can set the number of measures you require before the report file should be generated.

Download

Currently compiled as a static library for platform toolset Visual Studio 2012 - Windows XP (v110_xp). It is 32 bit, with flags /MD (Multi-threaded DLL).

Download link: benchmarklib-1.0.zip (md5 37ce3698a5af8b25555913f5151c85c9).

Using the Benchmark and Timer classes

Using the library

1. #include "benchmark.h"
2. Link against benchmarklib.lib

Using the MeasureCounts class

MeasureCounts counter(TimerFactory::Type::WindowsHRTimerImpl);

// Measure FPS
counter.setDescription("number of frames");
counter.setBatchSize(1000);

// Measure 30 times at 500 milliseconds interval (duration of 15 seconds)
counter.require(30);
counter.setBatchGroupSize(500);

// Results will be written to RESULT_EXAMPLE.TXT
counter.setOutput("EXAMPLE");

while ( ! counter.complete()) {

    /* do something */

    counter.measure();
}

Using the MeasureInterval class

MeasureInterval counter2(TimerFactory::Type::WindowsHRTimerImpl);

counter2.setDescription("milliseconds per frame");

// Measure 500 frames
counter2.require(500);

// Results will be written to RESULT_EXAMPLE2.TXT
counter2.setOutput("EXAMPLE2");

while ( ! counter2.complete()) {

    /* do something */

    counter2.measure();
}

Using the timer class

auto timer = TimerFactory::factory(TimerFactory::Type::WindowsHRTimerImpl);
timer->start();

/* do something */

std::cout << "Run time CPU : " << timer->end() << " milliseconds." << std::endl;

Example output (RESULT_EXAMPLE.TXT)

@@@ BENCHMARK @@@

profile "EXAMPLE"

measure "number of frames" / 1000 ms.

measure interval = 500 ms.


@@@ STATISTICS @@@

N          30
Mean       122.5427
S.E. Mean  0.2686
Std. Dev   1.4714
Variance   62.7829
Minimum    118.8119
Maximum    124.5059
Median     122.7723


@@@ HISTOGRAM @@@

                      #       
                      #       
                      #       
                      #       
                      #       
                      #       
                      #       
                #  #  #       
          #     #  #  #       
 #        #  #  #  #  #  #    
 #  #  #  #  #  #  #  #  #  # 
------------------------------
 0. 1. 2. 3. 4. 5. 6. 7. 8. 9.

0.  118.8119 <> 119.4446  =  2
1.  119.4446 <> 120.0772  =  1
2.  120.0772 <> 120.7099  =  0
3.  120.7099 <> 121.3426  =  3
4.  121.3426 <> 121.9752  =  2
5.  121.9752 <> 122.6079  =  4
6.  122.6079 <> 123.2406  =  4
7.  123.2406 <> 123.8733  =  11
8.  123.8733 <> 124.5059  =  2
9.  124.5059 <> 125.1386  =  1

Troubleshoot

Use this if your compiler does not support typed enums:

auto timer = TimerFactory::factory("WindowsHRTimer");

And for the benchmark classes as well.. (Edit: not yet supported for the Measure classes)

MeasureCounts counter("WindowsHRTimer");