Neither one nor Many

 
april 22 2014

Solving the puzzle at PHPBenelux 2014

Last conference at the Dutch PHP Conference (DPC) I wrote a summary (in Dutch). This time I couldn't find the time, I took notes though, but I was/am too lazy. For PHP Benelux 2014 experiences you can read on other people their blogs, for example the one David wrote at his blog. And if you're looking for sheets, chances are you can find them at the event's joind.in page.

I made one big mistake at PHPBenelux, that was missing (by accident (don't ask why :p)) Ross Tuck's talk. Was really curious about that one. Of the other talks I especially liked Bastian Hofmann's talk "Marrying front with back end". His talk at DPC was also very good IIRC.

Dutch web alliance puzzle

An old collegue of mine (Marijn) and I decided to programmatically solve the puzzle created for the event there DWA. You can find the puzzle instructions in the photo I took (see fig.2.) We heard a rumour that you could win an ipad if you solved it. Unfortunately that wasn't true img1. In short you have to place all cubes side-by-side in such a way that rotating them combined should yield four unique labels each rotation step. There is only one possible solution.

We decided both on different approaches and see who would solve it first. Challenge accepted! I lost unfortunately, I even had to abandon my first approach and made the switch to semi-brute-force as soon as I gained more insight in how it could be solved more easily this way. Marijn approached it brute force from the start.

I gave each cube a number {1, 2, 3, 4}, and each sidename a letter {a, b, c, d, x, y}. Next I defined for each cube three directions {a, b, c, d}, {x, b, y, d} and {x, c, y, a} (see fig.1.). All these directions can also be in reverse ({d, c, b, a}, ...).

fig.1. possible directionsfig.2. puzzle instructions

My first idea was to place a cube on a side randomly, choosing for each cube a random direction to rotate in, and try four rotation steps. If in all steps the pictures were unique that would be a solution. I made a mistake in the implementation, totally forgot that {x, c, y, a} is also a possible sequence. What also didn't help is that I misread the instructions at first, thinking all sides had to be equal (instead of not-equal), which isn't possible if you take a good look at the actual cubes. I guess beer + coding isn't always a good combination.

My second approach was simply, generate "layouts" for each cube's three directions, i.e. for {a, b, c, d}:

  • {a, b, c, d}
  • {b, c, d, a}
  • {c, d, a, b}
  • {d, a, b, c}
  • {d, c, b, a}
  • {c, b, a, d}
  • {b, a, d, c}
  • {a, d, c, d}

For each cube a, b, c, d mean different side types (the different logos, php, phpbenelux, dwa, phpelephant). I simply compare every combination, and filter out the duplicate results. You initially get eight solutions, because every solution you can start at the first, second, third or fourth "step" of the solution, and eight because of the extra "times two" multiplier you get for forward versus backward rotation.

Solver in C++

Code also available on http://ideone.com/ttFYMD.

#include <iostream>
#include <algorithm>
#include <stdexcept>
#include <string>
#include <sstream>
#include <vector>
#include <array>
#include <iomanip>

enum class sidename
{
    a, b, c, d, x, y
};

enum class sidetype
{
    phpbenelux, php, phpelephant, dwa
};

std::string sidename_str(sidename sn) {
    switch (sn) {
        case sidename::a: return "a";
        case sidename::b: return "b";
        case sidename::c: return "c";
        case sidename::d: return "d";
        case sidename::x: return "x";
        case sidename::y: return "y";
    }
    throw std::runtime_error{"invalid sidename"};
}

std::string sidetype_str(sidetype st) {
    switch (st) {
        case sidetype::phpbenelux: return "phpbenelux";
        case sidetype::php: return "php";
        case sidetype::phpelephant: return "phpelephant";
        case sidetype::dwa: return "dwa";
    }
    throw std::runtime_error{"invalid sidetype"};
}

/**
 * cubes define 6 side types (or pictures)
 *
 * cubes calculate for themselves all possible layouts, meaning
 * if you rotate them into some direction, you get a side, followed by side+1, side+2, side+3.
 * there are a few possibilities: reverse order, and in each layout (or "path") you can start
 * rotating the cube on side, side+1, side+2 or side+4 (starting point "shifts").
 */
class cube
{
public:

    cube(sidetype a, sidetype b, sidetype c, sidetype d, sidetype x, sidetype y)
        : a_{a}, b_{b}, c_{c}, d_{d}, x_{x}, y_{y},
          directionnames_({{ 
             {sidename::a, sidename::b, sidename::c, sidename::d},
             {sidename::x, sidename::b, sidename::y, sidename::d},
             {sidename::a, sidename::y, sidename::c, sidename::x} }}),
          directions_({{ {a, b, c, d}, {x, b, y, d}, {a, y, c, x} }})
    {
        for (int i=0; i<4; i++) {
            for (auto &sides : directions_) {
                // normal insert
                layouts_.push_back(sides);

                // reverse insert
                auto sidesrev = sides;
                std::reverse(std::begin(sidesrev), std::end(sidesrev));
                layouts_.push_back(sidesrev);

                // shift all
                sidetype temp = sides[0];
                for (int i=1; i<=3; i++)
                    sides[i - 1] = sides[i];
                sides[3] = temp;
            }
        }
    }

    const std::vector<std::array<sidetype, 4>> & layouts() { return layouts_; }

private:

/**
 * This is how I labeled each sidetype:
 * 
 *    X  =   a
 *  X X X   = x b y 
 *    X  =   c
 *    X  =   d
*/

    sidetype a_;
    sidetype b_;
    sidetype c_;
    sidetype d_;
    sidetype x_;
    sidetype y_;

    std::array<std::array<sidename, 4>, 3> directionnames_;
    std::array<std::array<sidetype, 4>, 3> directions_;
    std::vector<std::array<sidetype, 4>> layouts_;
};

/**
 * helper class that can see if a given solution is a duplicate from a previous solution
 * 
 * if you have a solution that is simply the same one, but rotating in a different direction
 * is not really a new solution. also note the four possible starting point in each layout/path.
 * so it will check if duplicates exist in both forward and backward directions, and for each
 * possible four shifts
 */
class solutions
{
public:
    solutions()
    {}

    bool is_dupe(std::array<std::array<sidetype, 4>, 4> temp2)
    {
        // Check if found solution isn't a duplicate
        bool duplicate = false;
        for (auto &solution : solutions_) {
            for (int j=0; j<8; j++) {
                duplicate = true;
                int sidenum = 0;
                for (auto &side : solution) {
                    auto &temp = temp2[sidenum++];

                    int count = 0;
                    int offset = j % 4;
                    if (j < 4) {
                        // Check if they are duplicates, as we use offsets of +0, +1, +2, +3 we can 
                        //  detect shifted duplicate results.
                        for (auto i = side.begin(); i != side.end(); i++) {
                            duplicate = duplicate && temp[(count + offset) % 4] == *i;
                            count++;
                        }
                    }
                    else {
                        // Check if they are duplicates simply in reverse order, also with the 
                        //  detect for shifted duplicates.
                        for (auto i = side.rbegin(); i != side.rend(); i++) {
                            duplicate = duplicate && temp[(count + offset) % 4] == *i;
                            count++;
                        }
                    }
                }
                if (duplicate)
                       return true;
            }
        }

        // Remember found solution, for duplicates checking
        solutions_.push_back(temp2);

        return false;
    }

private:
    std::vector<std::array<std::array<sidetype, 4>, 4>> solutions_;
};

int main (int argc, char *argv[]) 
{
/*
 * on the sheet:
 * 
 *    cube 4 (sideways)
 * 
 *   cube 1, 2, 3
*/

    cube one{
        sidetype::dwa,
        sidetype::phpelephant,
        sidetype::phpbenelux,
        sidetype::dwa,
        sidetype::php,
        sidetype::phpbenelux};

    cube two{
        sidetype::phpelephant,
        sidetype::phpbenelux,
        sidetype::phpbenelux,
        sidetype::phpbenelux,
        sidetype::php,
        sidetype::dwa};

    cube three{
        sidetype::phpbenelux,
        sidetype::dwa,
        sidetype::phpelephant,
        sidetype::php,
        sidetype::dwa,
        sidetype::phpelephant};

    cube four{
        sidetype::php,
        sidetype::phpelephant,
        sidetype::phpbenelux,
        sidetype::phpelephant,
        sidetype::dwa,
        sidetype::php};

    solutions solution;

    for (auto &cube1sides : one.layouts()) {
    for (auto &cube2sides : two.layouts()) {
    for (auto &cube3sides : three.layouts()) {
    for (auto &cube4sides : four.layouts()) {

        // Pictures have to be unique on each four cubes to be considered a unique solution..
        bool flag = false;
        for (int i=0; i<4; i++) {
            // .. Also on each four rotations of course
            flag = flag || (!(cube1sides[i] != cube2sides[i] &&
                              cube1sides[i] != cube3sides[i] &&
                              cube1sides[i] != cube4sides[i] &&
                              cube2sides[i] != cube3sides[i] &&
                              cube2sides[i] != cube4sides[i] &&
                              cube3sides[i] != cube4sides[i]));
        }
        if (!flag){
            // Skip duplicate solutions
            if (solution.is_dupe({cube1sides, cube2sides, cube3sides, cube4sides})) {
                continue;
            }

            // Print the result
            std::cout << "The cube-layout for the solution:" << std::endl << std::endl;

            static auto print = [](const std::string &cube, decltype(cube1sides) &sides) {
                std::cout << cube << ": " 
                    << " front: " << std::setw(15) << sidetype_str(sides[0]) << ", "
                    << " up: " << std::setw(15) << sidetype_str(sides[1]) << ", "
                    << " top:   " << std::setw(15) << sidetype_str(sides[2]) << ", "
                    << " back:  " << std::setw(15) << sidetype_str(sides[3])
                    << std::endl;
            };

            print("cube #1", cube1sides);
            print("cube #2", cube2sides);
            print("cube #3", cube3sides);
            print("cube #4", cube4sides);
        }
    }}}}
}

Output:

cube #1:    front:             php,     up:     phpelephant,    top:        phpbenelux,     back:              dwa
cube #2:    front:      phpbenelux,     up:             dwa,    top:       phpelephant,     back:              php
cube #3:    front:     phpelephant,     up:      phpbenelux,    top:               dwa,     back:      phpelephant
cube #4:    front:             dwa,     up:             php,    top:               php,     back:       phpbenelux

Performance is on my pc 0,00202 seconds on average per run (see perf.txt). The average is 0,00181 seconds, 11.60% faster, if we stop processing when we find the 'first' solution.

I still think my first idea should also work and I'm curious if that "random brute-force" algorithm would on average find a solution faster compared to finding the first solution in my current solver (in terms of required compares). img1.

Webdevelopment Comments (0)
 
februari 5 2014

Serving specific requests from a mutable cache

You can record requests now in a new tab "Cache", and you can choose to use this cache per request. Any such request will be immediately returned from the cache (instead of being proxied).

The cool thing is that you can modify the cache, so modifying CSS / Javascript is easy for example. I wanted this feature so I can debug some stuff on environments other than development. There are other ways to do this, but I think this is very convenient.

Some use cases

  • Change <script src=... includes to URL's from production to your dev environment, to simply test javascript on production pages.
  • Change javascript includes to their unminified / deobfuscated versions..
  • Add 'debugger' / console.log's to the source.
  • I've used it to more conveniently debug www.google-analytics.com/ga.js (an obfuscated javascript), by de-obfuscating it and using the step debugger in chrome.

By the way, a cool way to set a breakpoint in Javascript on specific function calls could be this. In ga.js there is a _gaq array, with a push function _gaq.push([something]);, I simply added this with wxhttpproxy in ga.js:

var foo = _gaq.push;
_gaq.push = function () {
    debugger;         <<<<<  and the debugger will intercept, you can view the stacktrace
    return foo.apply(_gaq, arguments);
}

Features / Roadmap

Note that the wxhttpproxy will intelligently modify the Content-Length header in case you modify the content. Content-Encodings gzip/deflate are not yet supported, for the time-being the proxy will garble the Accept header (by replacing gzip -> nogz, and deflate -> noflate). This is a workaround, that makes sure gzip/deflate isn't used.

Supported for chunked encoding should be added (easy), you can already edit it manually, just remove the chunked header, the hex numbers and add a Content-Length to work around it.

Real support for gzip / deflat should be added. I already have code for gzip / deflate support laying around somewhere.

Download

As an Ubuntu/debian package available here.

  • It uses wx2.8. In wx3.0 the event model has changed a bit, and I believe there is an error in the event handling w/regards to sockets. I will post some more details on this later, and if I have the time see if I can fix it..
Webdevelopment Comments (0)
 
december 26 2013

My super awesome NVIDIA Quadro K600 doesn't work properly with the default video drivers in Linux mint 15, 16 or Ubuntu 13.10. Especially in mint it was especially unstable. In Ubuntu everything seems fine for a few days, until the GPU finally crashed as well.

Linux mint 15 / 16

You disable the default driver called nouveau, to be loaded by the kernel with nouveau.blacklist=1.

In mint I've tried editing GRUB_CMDLINE_LINUX etc. in /usr/share/grub/default/grub and all of /etc/grub.d/*. Somehow update-grub didn't parse it, I was not so patient, so I ended up simply editting /boot/grub/grub.cfg.

trigen@Firefly21 ~ $ vim /boot/grub/grub.cfg
...
:g/^\s*linux
        linux   /boot/vmlinuz-3.8.0-19-generic root=UUID=60f8754f-f688-461e-b120-bf8402c1e2a9 ro nouveau.blacklist=1
        linux   /boot/vmlinuz-3.8.0-19-generic root=UUID=60f8754f-f688-461e-b120-bf8402c1e2a9 ro recovery nomodeset nouveau.blacklist=1
        linux16 /boot/memtest86+.bin
        linux16 /boot/memtest86+.bin console=ttyS0,115200n8
Press ENTER or type command to continue

Reboot the system. CTRL+ALT+F1, login, sudo service mdm stop, ./NVIDIA-Linux-x86_64-331.20.run and follow instructions. Reboot.

Ubuntu 13.10

In ubuntu I attempted to directly edit /boot/grub/grub.cfg again. Adding the blacklist parameter, somehow this failed, the NVIDIA installer still complaining about nouveau being loaded.

So I attempted the 'normal approach' again: vim /etc/default/grub, modified this line: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash nomodeset nouveau.blacklist=1". I also googled and found this answer on stackoverflow, suggesting the nomodeset is necessary as well. (So I added both). sudo update-grub and EAT REBOOT INSTALL REPEAT.

Reboot the system. CTRL+ALT+F1, login, sudo service lightdm stop, ./NVIDIA-Linux-x86_64-331.20.run and follow instructions. Reboot.

Some notes: Install grub into specific partition

This is a "Note to self", how to install grub for specific partition (source). I needed this command fixing my dual boot (linux and windows on one ssd).

sudo mount /dev/sda5 /mnt
sudo grub-install --root-directory=/mnt/ /dev/sda
sudo update-grub

Somehow linux mint f*cked up my boot to windows 8 partition. It had some problems recognizing my partition table or something. (At work I have the exact same setup, and there were no problems.) I ended up fixing it with the above command, and from windows (had to restore an image) using this tutorial that uses EasyBCD.

Linux/Unix Comments (0)
 
december 25 2013

In shell scripting I prefer the Kornshell. A while ago I experimented with "oh my zsh", but I switched back to ksh. Their auto completion for program commands is really unsurpassed (tab completion on program parameters for grep for example). The auto-incorrect however, is quite annoying img1. There is also a git plugin that visualized the active git branch in the $PS1 prompt. I liked these features and I want to add them to ksh.


Apparently I like old things

I fixed a few things for ksh in my .kshrc at bitbucket.

Separated history amongst the different ksh shells.

My problem with the default behaviour: by default history is globally shared amongst all shells. I tend to work in a screen and do different stuff in each buffer. So it's annoying if stuff from one buffer magically appears in the other buffer.

So what I do is I make sure there is one history file ~/.ksh_history which contains all history from all shells. When starting a new shell I copy this file into a history file specific for that ("sub")shell, i.e. ~/.ksh_history_files/<shellpid>. Each new shell does this.

When a shell is started all history files from ~/.ksh_history_files/* are merged back into ~/.ksh_history. And the ones that are no longer in use (shells have exited) are removed. This is done with a simple lsof call.

Commands are processed through a simply ksh function that makes sure all history lines are unique, without changing the order.

Some funny caveats were:

  • A history file should start with the character sequence \x81\01.
  • Each command in history file should end with a \x00 character.

A prompt that embeds current git branch

If you are inside a git clone).
Maybe the oh-my-zsh git integration is more advanced, no idea. Luckily this visualization in $PS1 is very fast.

trigen@Firefly21:/projects> cd smashbattle
trigen@Firefly21:/projects/smashbattle[network_multiplayer]> git branch
  master
* network_multiplayer
trigen@Firefly21:/projects/smashbattle[network_multiplayer]> 

Fix "cd" (with no params) to be the equivalent of "cd ~" again

Fixed with a simple alias around cd provided by /usr/share/ksh/functions/dirs. For dirs usage check here.

trigen@Firefly21:/usr/local/src/wxWidgets-3.0.0> cd <<<< this didn't work on Ubuntu 13.10 anyways.
trigen@Firefly21:/home/trigen> 

Integrated my "launcher tool" I wrote for windows a long time ago.

The browser is assumed to be "chromium-browser" (sudo apt-get install chromium-browser).


The result for gi dan flavin is opened in a new chromium tab.

  • i <url> - open url in browser
  • g <search terms> - search with google.com
  • gi <search terms> - search with google images
  • gv <search terms> - search with google videos
  • gnl <search terms> - search with google.nl
  • gs <search term> - search google scholar
  • w <search term> - search wikipedia
  • wa <search terms> - search wolfram alpha
  • yv <search terms> - search youtube
  • y <search terms> - search yahoo
  • yi <search terms> - search yahoo images
  • yv <search terms> - search yahoo videos
  • imdb <search terms> - search imdb
  • h <search term> - search hyperdictionary
  • v <search term> - search vandale (dutch dictionary)

Usage example: Open run menu (ALT+F2). Then type gi dan flavin to get the example result from screenshot.

I had to abandon my pure-ksh-functions approach to make the launcher commands available everywhere. So you have call install_launcher as root to install the shortcuts as scripts in /usr/local/bin/. Ubuntu does not respect shell functions in run unfortunately.

Linux/Unix Comments (0)
 
december 3 2013

As a free alternative for Charles proxy and/or Fiddler I developed wxHttpProxy in one (long) night! Unfortunately I had to spend the next day making it stable, and support https traffic as well img1

The mentioned alternatives are really great by the way, and have quite a few additional features, I just needed a simple http proxy that I could use in metalogmon, that's why I developed my own. It is opensource, find it on bitbucket.

The idea is to integrate this http proxy into metalogmon, so that you would be able to process your output through javascript. The reason that request/response are both first class citizens is that where I currently develop software I plan to use this proxy between all subsystems communication, and here multiple services communicate with eachother using REST calls.

I want to visualize the call sequences/hierarchy, like [A -> B -> C], [A <- B <- C]. And not group them together like: [A <> B], [B <> C].

Install binary (64 bit and ubuntu/debian based system only)

Install my key

wget -O - http://cppse.nl/apt/keyFile | sudo apt-key add -

Add my repo to apt's sources.list.

sudo sh -c "echo deb http://cppse.nl/apt/dists/stable/main/binary / >> /etc/apt/sources.list"

Install the package:

sudo aptitude update
sudo aptitude install wxhttpproxy

Usage

wxHttpProxy

Tip of the day, add a shortcut for starting the proxy (like WINKEY+2 or something)

Available shortcuts

  • alt+i - Toggle intercept mode (capture/display request/response, or be silent)
  • alt+c - Clear buffer(s)

Catch library

With Catch I developed a few helper classes in a test-driven way, it's a lightweight single-header c++ library for creating unit tests. Really liked working with it and look forward to using it in the future some more.

The apt repository

The shiny new apt repository was also really easy to setup, once you have made your packages, simply follow this wiki.

Other platforms

Should you wish to compile it for windows, it is wxWidgets 3.0.0 based and should already be cross-platform, just didn't test it yet. Use Dialogblocks to compile it (and also have it compile wxWidgets for you (as that's just convenient img1)).

C++ Comments (1)
 
oktober 20 2013

Implemented this feature I saw in XChat

Yes, having switched to Linux and using XChat, I now first realize how slow mIRC is (connecting to my bouncer, joining all kinds of channels, replaying scrollback, etc.-- mirc takes a few seconds, where XChat is "instant"), but I don't care. It's still my client of choice for Windows. On Tweakers IRC network, it is custom for a lot of people to indicate their away status with /nick <nickname>|afk and such. That makes sense, but XChat does something "smart", it periodically /who's all channels to know whom are away so it can color those in light grey in the nicklist. Cool feature I thought! So I "extended" my nicklisting coloring mirc script, by adding aways.ini. It's no longer necessary to keep changing your nickname with an away status (on freenode it isn't even allowed in some channels img1).


Away nicks in light-grey in XChat

The same feature in mIRC

It's a pity that such an inefficient implementation is required ("pull"ing /who periodically will result in a lot of extra network data), but the IRC protocol does not currently provide something that "push"es away info more intelligently. You can download the script here, note that aways.ini depends on nicklist.ini, also downloadable there. (nicklist.ini still works without aways.ini, it will then use light grey at random instead.)


Away nicks in light-grey


Away nicks in light-grey, other nicks colored.

Blog Comments (0)
 
oktober 7 2013

These are the ones I keep forgetting about:

  • Enabling line numbers: Tools, Options, Text Editor, All Languages, Show Line Numbers (source: http://stackoverflow.com/questions/5905687/can-i-see-line-numbers-in-the-vs2010-editor)
  • Toggle whitespace/tabs: CTRL+R, CTRL+W. (source: http://stackoverflow.com/questions/4065815/how-to-turn-off-showing-whitespace-characters-in-visual-studio-ide)
  • Navigate cursor to previous+next position: CTRL+- and forward CTRL+SHIFT+-.

These are a few you can't do without:

  • Step into function: F12 (CTRL+- to return cursor to previous position)

If you can afford it you can't do without Visual Assist X either.

C++ Comments (0)
 
september 1 2013

Live video streaming

I was interested in specifically the generation of H264 videos directly from code (without audio). This was something I used to do by generating separate .BMP files that I then convert with mencoder. This costs quite a lot of hdd space and was performance-wise not optimal. Of course I already figured that out, but figuring out how to encode directly to video was something on my To Do list.

Believe it or not, I actually had to search a while for libraries and SDK's, before concluding that ffmpeg was the best option. I've had a bad experience with ffmpeg 5 years ago, so I was probably ignoring ffmpeg in the google search results. You can link to ffmpeg's libraries and use them in your own program, they also provide code samples. This made it all pretty easy to setup. So when I completed generating H264 videos from code I wondered how difficult it would be to directly stream video with ffmpeg.

One cool application would be to use it in games, f.i. make a small livestream for actual games being played on the server. img1

Ffmpeg supports a few protocols, I chose an RTMP server for my setup, which is Flash only. If that works it'll probably work for other protocols as well. The best alternative for HTML5 is probably Apple's HLS Streaming, which actually is pretty awesome, as a side-note from source:

If multiple qualities of a stream are available, the player will continuously monitor the current bandwidth and pick the next fragment from the highest quality it can load. This is called Adaptive Streaming:

For the videoplayer I chose JWplayer version 6.

Setting up an rtmp server with Jwplayer

The servers I tested: rtmplite on linux, crtmpserver on linux and Red5 Media Server on windows. The linux server I used was a Suse (SLED11) micro EC2 instance on Amazon AWS. But these should all work on linux and windows.

RTMP is flash-only, traffics over port 1935.

Some notes on using rtmplite with JWplayer

Eventually I chose not to use rtmplite, but crtmpserver instead, as you have more output there as to what's going on in the server. rtmplite output is less useful to me, by default it outputs nothing, with verbose you get messages at packet level. img1 rtmplite used quite a bit more of my CPU compared to crtmpserver (not very accurate due to EC2, but crtmpserver at 0.3% CPU versus > 10% CPU for rtmplite) and it does not understand aspect ratio other than 4:3, so 16:9 will be "squished" into 4:3. It's not my intent to be negative about rtmplite, if you're a python guy It's probably very easy to tweak and I must say it works out-of-the-box with a simple python rtmp.py, ready for video at rtmp://servername/app/video.

With JWplayer you can view the stream with:

jwplayer("myElement").setup({
    file:       "rtmp://localhost/appflv:video",
    autostart:  true,
    width:      480,
    height:     320
});

If you attempt to use that config, please note that file URL! It took me an hour to get that correct, in all JWPlayer samples you see rtmp://server/app/flv:video.flv or rtmp://server/app/mp4:video.mp4. The prefix flv: or mp4: being just for JWplayer so that it knows what encoding to expect. As the .mp4 or .flv extension in URL's is not mandatory. Eventually JWplayer will strip away flv:. An rtmp url like rtmp://server/app/video consists of an application called app and a video called video. Stripping away the prefix from rtmp://cppse.nl/app/flv:video will provide app/ for the application which looks like app but isn't quite the same. img1

So... crtmpserver then!

After compiling you fire it up with default settings with: ./crtmpserver/crtmpserver crtmpserver/crtmpserver.lua. You should see something similar to the following output.

ip-10-224-83-43:/usr/local/src/crtmpserver/builders/cmake # ./crtmpserver/crtmpserver ./crtmpserver/crtmpserver.lua
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:203 Initialize I/O handlers manager: epoll
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:206 Configure modules
/usr/local/src/crtmpserver/sources/thelib/src/configuration/module.cpp:84 Module /usr/local/src/crtmpserver/builders/cmake/applications/appselector/libappselector.so loaded
/usr/local/src/crtmpserver/sources/thelib/src/configuration/module.cpp:84 Module /usr/local/src/crtmpserver/builders/cmake/applications/flvplayback/libflvplayback.so loaded
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:212 Plug in the default protocol factory
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:219 Configure factories
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:225 Configure acceptors
/usr/local/src/crtmpserver/sources/thelib/src/netio/epoll/iohandlermanager.cpp:100 Handlers count changed: 0->1 IOHT_ACCEPTOR
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:231 Configure instances
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:237 Start I/O handlers manager: epoll
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:240 Configure applications
/usr/local/src/crtmpserver/sources/thelib/src/configuration/module.cpp:177 Application appselector instantiated
/usr/local/src/crtmpserver/sources/thelib/src/configuration/module.cpp:177 Application flvplayback instantiated
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:246 Install the quit signal
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:257
+-----------------------------------------------------------------------------+
|                                                                     Services|
+---+---------------+-----+-------------------------+-------------------------+
| c |      ip       | port|   protocol stack name   |     application name    |
+---+---------------+-----+-------------------------+-------------------------+
|tcp|        0.0.0.0| 1935|              inboundRtmp|              appselector|
+---+---------------+-----+-------------------------+-------------------------+
/usr/local/src/crtmpserver/sources/crtmpserver/src/crtmpserver.cpp:258 GO! GO! GO! (19664)

With the default configuration anyone can stream a video to the RTMP server, as inbound and outbound are over port 1935. I tried separating that with the configs, but I couldn't find any configuration setting to do this, so I hacked something in the source to only allow inbound RTMP for 127.0.0.1 (see patch.txt). With crtmpserver you don't have the URL problem in JWPlayer.

jwplayer("myElement").setup({
    file:       "rtmp://server/flvplayback/flv:video",
    autostart:  true,
    width:      480,
    height:     320
});

Sending video to the server with "sendstream"

The proof-of-concept tool I set out to make is now a useful test tool you can use to stream a test video with optional audio at a given resolution to a given streaming server. I have compiled it for windows and linux 32 bit (opensuse 12) and also have that in a working chroot for download (~28 mb zipped).

usage: sendstream.exe < RTMP URL > [ < width > < height > < sound > < max_fps >]
 params: width default = 320, height default = 240 pixels, sound default = 1, max_fps default = 25.
 i.e: sendstream.exe "rtmp://localhost/app/video" 800 600

Example output for sendstream.exe rtmp://cppse.nl/flvplayback/video 1920 1080 1:

[libmp3lame @ 058a6320] Channel layout not specified
Output #0, flv, to 'rtmp://cppse.nl/flvplayback/video':
    Stream #0:0: Video: flv1, yuv420p, 1920x1080, q=2-31, 100 kb/s, 90k tbn, 20
tbc
    Stream #0:1: Audio: mp3, 44100 Hz, 2 channels, s16p, 24 kb/s
HandShake: client signature does not match!
writing frame 0 at 0.044793... +V
writing frame 1 at 0.112187... +A+A+A[flv @ 058a2b60] Encoder did not produce pr
oper pts, making some up.
+A+V
writing frame 2 at 0.324518... +A+A+A+V
writing frame 3 at 0.813168... +A+A+A+A+A+A+A+A+A+V
writing frame 4 at 1.03129... +A+A+A+A+A+A+A+A+A+A+A+A+A+A+A+A+A+A+A+V
writing frame 5 at 1.16484... +A+A+A+A+A+A+A+A+A+V
writing frame 6 at 1.23609... +A+A+A+A+A+V
writing frame 7 at 1.28777... +A+A+A+V
writing frame 8 at 1.33955... +A+A+V
writing frame 9 at 1.38883... +A+A+V
writing frame 10 at 1.43901... +A+V
writing frame 11 at 1.48954... +A+A+V
writing frame 12 at 1.53965... +A+A+V
writing frame 13 at 1.58858... +A+A+V
writing frame 14 at 1.6456... +A+A+V
writing frame 15 at 1.69497... +A+A+V
writing frame 16 at 1.74456... +A+A+V

Audio and video are combined with interweaving and if rendering of video frames is relatively slow compared to audio it may require more audio frames in between and vica versa. In the output +A is a written audio and +V a written video frame.

I use SFML for generating the graphics and for audio something similar to the example code by ffmpeg. The interweaving part was the most difficult but important thing to get right. This is the easiest version of the code I could come up with that works properly. img1 Something I didn't know in the beginning was that you don't need to keep your framerate of sending frames to the server consistent at a given frames per second-rate.

sfml_init(&pixels, &pixelSize);

auto timer = TimerFactory::factory(TimerFactory::Type::WindowsHRTimerImpl);
timer->start();

frame->pts = timer->end();

int64_t currentframe = 0;
while (true) {

    sfml_generate_frame();

    std::cout << "writing frame " << currentframe++ << " at " << (timer->end() / 1000.0) << "... ";

    // Make sure to write enough audio frames until the audio timer lines up
    while (true) {
        double audio_pts = (audio_st) ? (double)audio_st->pts.val * audio_st->time_base.num / audio_st->time_base.den : 0.0;
        double video_pts = (video_st) ? (double)video_st->pts.val * video_st->time_base.num / video_st->time_base.den : 0.0;

        if (audio_pts >= video_pts)
            break;

        std::cout << "+A";
        write_audio_frame(oc, audio_st);
    }

    // Write video frame
    write_video_frame(oc, video_st);
    std::cout << "+V" << std::endl;

    // Set elapsed time as frame time
    frame->pts = timer->end();
}

One final side-note if you are going to use sendstream on linux with Xvfb (X virtual framebuffer). On SUSE it was a pain to get it working, I ended up installing it on an openSuse VMWare VM where it did work and then made a minimal chroot for it, one that includes Xvfb and sendstream and a run.sh script (you might want to edit that). Not all screen resolutions you can think off that work fine with RTMP server will work, i.e.: I wanted a 980 x 100 resolution video, that didn't work for Xvfb, it does work if you create a larger screen with a more common resolution like 1280 x 768. Also 980 x 100 didn't work for SFML, 1000 x 100 did, probably the OpenGL underneath complaining or something. AFAIK these problems do not exist without Xvfb.

The result

Sendstream runs on my webserver capped at 2 FPS and without audio, so that it doesn't hurt the CPU too much.

Download: sendstream-1.0.zip for windows. 11.2MB

Download: sendstream-1.0-chroot.zip for linux (chroot, use: "chroot /path/to/sendstream /run.sh"). 27.2MB

a930805b8867232755da195d34587476 *sendstream-1.0-chroot.zip
6ed8cf5102dbe10c5f4722c80d38cbbd *sendstream-1.0.zip

 

C++ Comments (0)
 
juni 28 2013

Het thema van de Dutch PHP Conference 2013 was Respecteer het eco-systeem.

Dag één begon voor mij (en mijn collega's) 7 juni en Ade Oshineye opende met het pleiten voor User Journeys, de flows over meerdere devices die gebruikers volgen zoals openen van een nieuwsbrief op een telefoon en naar doorklikken naar product om deze uiteindelijk te kopen op een werkstation.

User Journey

Denk ook na over dat een website geopend in een LinkedIn app nog kleiner getoond wordt (aangezien het in een webview geladen wordt, soort een frame). Hoe ziet je site er dan nog uit? Hoe vaak komt het niet voor dat je vanuit een nieuwsbrief door wilt klikken naar een product om vervolgens geconfronteerd te worden met een inlogscherm? Kan dat niet gebruiksvriendelijker?

Unbreakable Domain Models van Mathias Verraes bevatte tips die mij doen denken aan OO in C++: goed gebruik maken van types. I.p.v. een string voor een e-mail echt een Email class maken die eventueel ook validatie regelt. Wat ik interessant vind om te zien is dat de spreker in zijn sheets gebruik maakt van exceptions in de constructor. Waar ik persoonlijk ook groot fan van ben maar veel mensen in mijn omgeving niet img1. Naar mijn mening moet je een type niet half kunnen aanmaken (constructen) want dan is het geen volledig type, maak dan een liever een ander type die die state representeert. Zijn slides over "Encapsulate operations" tonen gebruik van een CustomerSpecification interface en het maken van classes zo dicht mogelijk bij het domein.

Surviving a Prime Time TV Commercial - David Zuelke. Zeer interessante manier van vertellen met veel tips en info, triggert mij te kijken naar e.e.a., zoals genoemde supervisord, transactional e-mail of elastic search. Voor hun use-case was het zinvol om zo'n beetje alles in Elastic Search op te slaan.

PHP, Under The Hood - Anthony Ferrara. Ook goede spreker en presentatie was humorvol. Hij heeft een PHP compiler gemaakt in PHP en wist leuke dingen te vertellen over opcodes en liet bijbehorende code hier en daar zien. Helaas talk maar voor de helft gezien. Dat vm_execute.h gegenereerd wordt tijdens compileproces door PHP vind ik niet zo schokkend, dat je heel g++ compiled met g++ vind ik pas schokkend! img1

Mikado!

Uncon: The Mikado method - Pascal de Vink (slides here). Hele korte talk, maar wel interessant. Een methode omtrent een leuk idee, met als onderliggende toon misschien wel het tegenovergestelde van "Respecteer het eco-systeem":

  • Formuleer een doel, schrijf deze op papier.
  • Naief implementeren van je doel in de code, 'fuck' de details of de dingen die je stuk maakt. Gewoon coden hoe het moet worden.
  • Als je dit foutloos hebt gedaan en alles werkt nog: klaar. Maaarrr waarschijnlijk heb je wel een aantal dingen stuk gemaakt, heb je een lijstje compile errors of heb je in je hoofd een aantal TODO's of FIXME's img1
  • Je schrijft deze als dependencies onder je doel (alles met pen en papier). Dit zijn dan de prerequisites voor je doel in een boomstructuur.
  • Je fixed ze als het simpel is of shelved je changes en gaat één voor één die prerequisites fixen en comitten.
  • Je streept alles van je blaadje af en werkt zo gericht naar je doel toe.

Idee achter deze methode is dat je wellicht code weg moet gooien of shelven, maar daar tegenover staat dat je snel je prerequisites inzichtelijk hebt. Deze kun je ook aan een andere developer geven als een duidelijkere roadmap voor implementatie van dit doel.

Dependency Injection Smells - Matthias Noback. Volgens mij vond ik dit een saaie talk, weet het niet meer zeker. Maar wel veel herkenbare issues qua design en soort van eigenaardigheden in PHP. Zo is in PHP public function foo(Bar $baz) alleen maar afdwingen dat $baz de interface Bar implementeert. Maar is $baz gewoon de implementatie, en als je niet uitkijkt programmeer je tegen de implementatie aan i.p.v. de interface. (Gelukkig is phpstorm slim genoeg om dit snel herkenbaar te maken)

Getting your toolbox together - Daan van Renterghem. Was een leuke talk, ging over Vagrant. Persoonlijk nooit een hobby van me geweest het configureren van dit soort dingen, maar hij liet Vagrant zien en vertelde over Chef en dat was zeker wel interessant.

Mariokarten (N64)

De volgende dag...

Growth Hacking for Humans - Eamon Leonard. (video) Zeer interessante keynote. Een man die behoorlijk ondernemend is en een aantal fasen heeft doorlopen die hij wilde delen met de rest. Benadrukte de belangrijke rol van communities.

Scenario Driven API Design - Ivo Jansch. Sta je op het punt een (REST) API te ontwikkelen, neem dan deze talk even door. Leuke anecdote, spreker: "Wie heeft er wel eens wat gehoord over Scenario driven design, buiten deze talk om dan natuurlijk?". Eén of twee mensen steken hun vinger op. "Oh, dat is grappig want ik heb die term dus zelf verzonnen.". img1

Introduction to Django - Travis Swicegood. Was eerlijk gezegd best nieuwsgierig naar dit framework. Omdat ik binnenkort ook eens wat met Python wil gaan doen. Wel veel goede dingen over Django gehoord maar de spreker heeft niet echt goed de kracht van het framework over weten te brengen naar mij. Bleef overigens ook allemaal erg basic en na elke slide kwam een slide met daarop: "Are there any questions?".

Emergent Design with phpspec - Marcello Duarte. Deze meneer was er erg handig mee, hij is dan ook de man achter het project. Het is denk ik PHPUnit done right, maar dan niet zoals het ellendige SVN "CSV done right" zou moeten zijn. Het is iets breder, je werkt meer vanuit de specificaties (die geformuleerd zijn als tests). phpspec is gefocust om zo zinvol mogelijke foutmeldingen/feedback te geven en slimmer te zijn. Dat laatste wil zeggen dat het gespecte methoden of classes voor je kan aanmaken die nog niet bestaan. Het ondersteund je beter in Test of Behaviour driven development.

Measuring and Logging Everything in Real Time - Bastian Hofmann. Veel zinvolle tips m.b.t. logging. Veel voorkomende problemen is dat je meerdere servers en dus logs hebt, welke log entries overal horen nou bij elkaar. PHP heeft leuke apache_note. Andere trefwoorden: graylog2, elastic search, AMQP, logstash, graphite, statsd, boomerang.js, X-trace-id, monolog, error_log, set_error_handler, set_exception_handler. Zat een heel verhaal omheen, wat ik hier niet zal reproduceren.

Conclusie

Worse Is Better, for Better or for Worse - Kevlin Henney. Zeer boeiende spreker, kan ik echt lang naar luisteren zulke talks. Alan Kay meerdere keren geciteerd gedurende deze conference, zo ook in deze talk, zie ook phpspec slides, slide 40. Het messaging aspect tussen de objecten is onderbelicht geweest, zie slides 13, 14, 15 (erg leuk). Ik vind het "jammer" dat ik geen ervaring heb met COM, dus zal eens een keer uit nieuwsgierheid naar kijken. Hij ging in deze talk in op een uitspraak "Worse is better":

Over two decades ago, Richard P Gabriel proposed the thesis of "Worse Is Better" to explain why some things that are designed to be pure and perfect are eclipsed by solutions that are seemingly limited and incomplete. This is not simply the observation that things that should be better are not, but that some solutions that were not designed to be the best were nonetheless effective and were the better option. We find many examples of this in software development, some more provocative and surprising than others. In this talk we revisit the original premise and question in the context of software architecture.

Wat is het antwoord op deze vraag? Als Smalltalk zoveel "mooier" is, waarom programmeren we nu inmiddels niet allemaal in Smalltalk? Waarom werd C++ zo populair? Het antwoord is dat C++ het ecosysteem respecteerde. In de tijd van Smalltalk waren thuis computers nog zo snel niet en smalltalk was een stuk zwaarder, daarbij draaide het afgezonderd in een virtual machine in zijn soort van eigen ideale universum. Hierbuiten smalltalk is echter een heel eco-systeem wat in dit geval een "trage" computer is met een printer, muis, toetsenbord, etc. De spreker vertelde over hoe practisch hij Turbo C++ compiler vanaf een floppy op zijn computer zette en daarmee gelijk kon programmeren en echt dingen voor elkaar kon krijgen en dat de geschreven programma's snel waren. Dit is zijn waarheid, maar ik denk dat de kern van het ecosysteem respecteren wel een goed punt is, en voor mij zeer herkenbaar.

Om terug te komen op de keynote van dag één, waarom is het web zoals we dat nu kennen, met HTTP + HTML zo succesvol geworden? Er waren ook daar (geavanceerdere?) protocollen of in iedergeval alternatieven, zoals gopher. De rede is natuurlijk dezelfde, HTML "embraces the ecosysteem", het groeit mee met de ontwikkelingen, integreert en past zich aan. Denk aan Java applets, Flash, talloze file formats, VRML, WebGl maar ook juist integreert met andere protocollen zoals FTP--en ook al weet ik niet--vast ook gopher!

Blog Comments (0)

Paging
Page 1 <<<< You are Here!
Page 2
Page 3
Page 4
Page 5
Page 6
Author:
Ray Burgemeestre
february 23th, 1984

Topics:
C++, Lisp, Music, Philosophy

Other interests:
Music, Art, Zen, Webdevelopment