xenogenesi::blog
memento
3d adt android apache2 app apt aria2 build bullet cflags chromium codeigniter debian demoscene dependencies dpkg driver emulator freeglut gcc gfx git glut htaccess javascript json kernel linux make metalink minimal mysql opengl php python raspbian realtime rpi specs template toolchain update-alternatives video wifi wordpress

build ffmpeg with cuda support on debian

get ffmpeg sources and install build dependencies

mkdir ffmpeg-cuda
cd ffmpeg-cuda
apt-get source ffmpeg
sudo apt build-dep ffmpeg

patch the debian/rules to enable cuda (cuda dev packages need to be installed)

--- ffmpeg-3.4.2/debian/rules.orig      2018-03-18 00:08:00.717540941 +0100
+++ ffmpeg-3.4.2/debian/rules   2018-03-18 00:11:53.506937275 +0100
@@ -187,6 +187,8 @@
                --target-os=$(DEB_HOST_ARCH_OS)
 endif

+CONFIG += --enable-cuda --enable-cuvid --enable-nvenc --enable-nonfree --enable-libnpp --enable-cuda-sdk
+
 # Use the default debhelper scripts, where possible.
 %:
        dh $@

build it

cd ffmpeg-3.4.3
time nice dpkg-buildpackage -rfakeroot -uc -b -j5

the build will generate a bunch of *.deb files, they will conflict with the one installed by the system, so

mkdir install
for f in *.deb; do dpkg -x $f install; done

create a wrapper to use the built libraries (ffmpeg-cuda)

#!/bin/sh
export LD_LIBRARY_PATH=$INSTALL_PATH/usr/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH
exec $INSTALL_PATH/usr/bin/ffmpeg "$@"

use it to encode, for instance

ffmpeg-cuda -hwaccel cuvid -c:v h264_cuvid -i "$1" -vf scale_npp=1280:720 -c:v h264_nvenc "$2"
ffmpeg-cuda -i "$1" -c:v h264_nvenc "$2"

debian/stretch, apache2, mod_rewrite and .htaccess

by default debian/stretch apache2 installation have mod_rewrite disabled (at least the one used with debootstrap and lxc containers) so few steps are needed to enable it and .htaccess files

enable mod_rewrite (apache2 must be restarted, see below)

a2endmod rewrite

in the virtual host configuration enable .htaccess (apache2 must be restarted, see below)

<Directory "/var/www/htdocs">
     AllowOverride All
</Directory>

optionally enable mod_rewrite log (in the virtual host configuration) output goes into error.log (don’t leave it enabled in production) note: rewrite engine may need to be enabled before (apache2 must be restarted, see below)

RewriteEngine On
LogLevel alert rewrite:trace6

restart apache

systemctl restart apache2

SSL Let’s encrypt on Debian stretch with dehydrated

Assuming you’ve apache2 already installed and a reachable internet server running

apt-get install dehydrated

Add/edit /etc/apache2/conf-available/letscrypt.conf (notice the difference between the alias and the real directory which is plural acme-challenges):

Alias /.well-known/acme-challenge/ "/var/lib/dehydrated/acme-challenges/"
<Directory "/var/lib/dehydrated/acme-challenges/">
        Options None
        AllowOverride None
        # Apache 2.x
        <IfModule !mod_authz_core.c>
                Order allow,deny
                Allow from all
        </IfModule>
        # Apache 2.4
        <IfModule mod_authz_core.c>
                Require all granted
        </IfModule>
</Directory>

Add/edit /etc/dehydrated/conf.d/99_email.sh:

CONTACT_EMAIL="anymail@yourdomain"

Edit /etc/dehydrated/domains.txt, set the domains for which generate certificates.

Restart apache2 and run dehydrated -c, it should create the domain certs.

Edit your apache’s host file (/etc/apache2/sites-available/default-ssl.conf), replace DOMAIN with your domains:

SSLCertificateFile /var/lib/dehydrated/certs/DOMAIN/fullchain.pem
SSLCertificateKeyFile /var/lib/dehydrated/certs/DOMAIN/privkey.pem

Add/edit /etc/cron.daily/dehydrated:

#!/bin/sh

exec /usr/bin/dehydrated -c >/var/log/dehydrated-cron.log 2>&1

Run chmod 0755 /etc/cron.daily/dehydrated

Add/Edit /etc/logrotate.d/dehydrated:

/var/log/dehydrated-cron.log
{
        rotate 12
        monthly
        missingok
        notifempty
        delaycompress
        compress
}

lxc – linux containers

Some quick note.

install create run lxc container

sudo apt-get install lxc debootstrap

create from mirror with debootstrap

sudo MIRROR=http://httpredir.debian.org/debian lxc-create -n sid64 -t debian -- -r sid -a amd64

create a container from lxc template (default jessie, the host is 64)

sudo lxc-create -t /usr/share/lxc/templates/lxc-debian -n debian

create a container from lxc debian template, pass lxc-debian argument after --

sudo lxc-create -t /usr/share/lxc/templates/lxc-debian -n debian2 -- -r stretch

-d daemon (background)

sudo lxc-start -n sid64 -d 
sudo lxc-stop -n sid64

-F foreground

sudo lxc-start -n sid64 -F

change root password

sudo lxc-attach -n sid64 passwd

list container->dhcp ip

sudo lxc-ls --fancy

mount host directory inside the container

lxc.mount.entry=/host/dir /var/lib/lxc/{container-name}/rootfs/dir none bind 0 0

create a copy of the container using overlayfs

sudo lxc-copy -s -B overlayfs -n {base-container} -N {new-container-name}

install and enable the bridge

sudo apt-get install libvirt-clients libvirt-daemon-system ebtables dnsmasq

sudo virsh net-info default

# /var/lib/libvirt/network/default.xml
sudo less /etc/libvirt/qemu/networks/default.xml

always same ip by mac addr

<dhcp>
  <range start="192.168.122.100" end="192.168.122.254" />
  <host mac="00:FF:AA:00:00:01" name="foo.example.com" ip="192.168.122.101" />
  <host mac="00:FF:AA:00:00:02" name="bar.example.com" ip="192.168.122.102" />
</dhcp>

sudo virsh net-start default
/sbin/ifconfig -a

sudo vi /var/lib/lxc/{container-name}/config

lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = virbr0
lxc.network.hwaddr = 00:FF:AA:00:00:01
lxc.network.ipv4 = 0.0.0.0/24

bridge autostart (optional)

sudo virsh net-autostart default
sudo virsh net-info default

use host br0 instead of virsh lxc-nat

configure host br0 and add eth0 to it, /etc/network/interfaces:

auto br0
iface br0 inet static
    address 192.168.2.210
        netmask 255.255.255.0
        network 192.168.2.0
        broadcast 192.168.2.255
        gateway 192.168.2.1
    bridge-ifaces eth0
    bridge-ports eth0
    up ifconfig eth0 up

iface eth0 inet manual

edit /var/lib/lxc/{container}/config:

lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0
lxc.network.hwaddr = 00:FF:AA:00:00:10
lxc.network.ipv4 = 192.168.2.20/24
lxc.network.ipv4.gateway = 192.168.2.1

references: 1 2 3 4

apache2/php/mysql (logs) timezone

Setting timezone for apache/php/mysql (logs timestamp also)

# apache2 on debian (it get system tz)
dpkg-reconfigure tzdata

# this doesn't work (maybe useful for .htaccess/vhosts?)
echo "SetEnv TZ Europe/Rome" > /etc/apache2/conf-available/tz-rome.conf
service apache2 reload

# php
echo 'date.timezone = "Europe/Rome"' > /etc/php5/apache2/conf.d/30-tz-rome.ini

# mysql (untested)
default-time-zone = "Europe/Rome"

Debian update-alternatives

reference: askubuntu

First erased the current update-alternatives setup for gcc and g++:

sudo update-alternatives --remove-all gcc 
sudo update-alternatives --remove-all g++

Install Packages

It seems that both gcc-4.3 and gcc-4.4 are installed after install build-essential. However, we can explicitly install the following packages:

sudo apt-get install gcc-4.3 gcc-4.4 g++-4.3 g++-4.4

Install Alternatives

Symbolic links cc and c++ are installed by default. We will install symbol links for gcc and g++, then link cc and c++ to gcc and g++ respectively.

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.3 10
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.4 20

sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.3 10
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.4 20

sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc 30
sudo update-alternatives --set cc /usr/bin/gcc

sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++ 30
sudo update-alternatives --set c++ /usr/bin/g++

Configure Alternatives

The last step is configuring the default commands for gcc, g++. It’s easy to switch between 4.3 and 4.4 interactively:

sudo update-alternatives --config gcc
sudo update-alternatives --config g++

raw Debian package con tar e ar

apt-fast

Makefile:

all: deb

deb:
    mkdir -p tmp/usr/bin tmp/DEBIAN tmp/etc tmp/usr/share/man/man5 tmp/usr/share/man/man8
    find tmp -type d | xargs chmod 755
    cp apt-fast.conf tmp/etc/
    cp man/apt-fast.8 tmp/usr/share/man/man8/
    cp man/apt-fast.conf.5 tmp/usr/share/man/man5/
    cp apt-fast tmp/usr/bin/
    tar c --owner root --group root -v -z -f tmp/data.tar.gz -C tmp ./etc ./usr
    cd tmp && find usr/ etc/ -type f | xargs md5sum >DEBIAN/md5sums
    awk '/^#Package:/{i++}i {print substr($$0,2); }' Makefile >tmp/DEBIAN/control
    tar c --owner root --group root -v -z -f tmp/control.tar.gz -C tmp/DEBIAN ./control ./md5sums
    echo 2.0 >tmp/debian-binary
    ar rcv apt-fast_1.8-1.deb tmp/debian-binary tmp/control.tar.gz tmp/data.tar.gz

.phony: clean
clean:
    -rm -fr tmp

#Package: apt-fast
#Version: 1.8
#Section: net
#Priority: optional
#Architecture: all
#Essential: no
#Installed-size: 100
#Maintainer: Name <mail here>
#Source: https://github.com/ilikenwf/apt-fast
#Depends: aria2
#Description: apt-get wrapper with aria2 as backend downloader
# apt-fast is a shellscript wrapper for apt-get and aptitude
# can drastically improve apt download times
# by downloading packages in parallel
# with multiple connections per package.

compilare chromium in Debian

mkdir -p ~/src/chromium-deb
cd ~/src/chromium-deb
apt-get source chromium
sudo apt-get build-dep chromium 
time dpkg-buildpackage -rfakeroot -uc -b -j4      # DEB_BUILD_OPTIONS=nostrip 

28439,94s user 6571,60s system 280% cpu 3:28:10,43 total

du -sh ~/src/chromium-deb
12G

compiling take an average of 2G RAM or less but the final link is critical 3.5G+

using aria2 with apt-get

source: Make apt-get MUCH faster using aria2!
see also: apt-metalink

sudo apt-get install aria2

apt-get -y --print-uris -qq upgrade|python2.6 apt2metalink.py > tmp.meta4

sudo aria2c -d /var/cache/apt/archives/ -M tmp.meta4 --file-allocation=none

apt2metalink.py:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# reference: http://ubuntuforums.org/showthread.php?t=1493421
import sys
import re

if __name__ == "__main__":
    entries = []
    while True:
        line = sys.stdin.readline()
        if not line: break
        uri, name, size, chksum = line.split()
        uri = re.sub(r"^'|'$", "", uri)
        hashfunc, hashvalue = chksum.split(':')
        hashfunc = hashfunc.lower()
        if hashfunc == 'sha256':
            hashfunc = 'sha-256'
        entries.append((uri, name, size, hashfunc, hashvalue))

    print """<?xml version="1.0" encoding="UTF-8"?>
    <metalink xmlns="urn:ietf:params:xml:ns:metalink">"""

    for e in entries:
        print """<file name="{name}">
    <size>{size}</size>
    <hash type="{hashfunc}">{hashvalue}</hash>
    <url priority="1">{uri}</url>
    </file>""".format(uri=e[0],name=e[1], size=e[2], hashfunc=e[3], hashvalue=e[4])
    print """</metalink>"""

one shell script:

#!/bin/sh -e
apt-get -y --print-uris -qq upgrade|awk '
    BEGIN {
      print "<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
      print "<metalink xmlns=\"urn:ietf:params:xml:ns:metalink\">"
    }
    { gsub(/\x27/,"",$1);
      split($4, chksum, /:/)
      printf "<file name=\"%s\">",$2
      printf "<size>%d</size>", $3
      printf "<hash type=\"%s\">%s</hash>", chksum[1], chksum[2]
      printf "<url priority=\"1\">%s</url>", $1
      print "</file>"
    }
    END {
      print "</metalink>"
    }'|aria2c -M- --file-allocation=none -d /var/cache/apt/archives/