PAB's blog

mercredi 27 juin 2018

Certbot "ImportError: cannot import name jose" on Debian Stretch

Par PAB le mercredi 27 juin 2018, 08:46

I recently ran into the following error when running `certbox renew`:

Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==0.19.0', 'console_scripts', 'certbot')()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 561, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2631, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2291, in load
    return self.resolve()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2297, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python2.7/dist-packages/certbot/main.py", line 11, in <module>
    from acme import jose
ImportError: cannot import name jose

I fixed it by launching the following command:

apt-get install certbot python3-certbot python3-acme python3-cryptography python3-josepy python3-setuptools -f -t stretch-backports

lundi 18 juin 2018

Gitlab behind a reverse proxy handling https

Par PAB le lundi 18 juin 2018, 14:58

I share below my configuration of GitLab when served behind a reverse proxy (haproxy to name it) that is handling the HTTPS part of the communication (and is also load balancing). <Disclaimer> I am not claiming this is the best configuration or the only possible configuration but I can report that it works well</Disclaimer>.

Infrastructure

The network transaction is established as follows:

User <- HTTPS HTTP/2 -> HAPROXY <- HTTP -> NGINX <- HTTP -> Unicorn/GitLab

I will not detail here the configuration of HAProxy which is pretty classical.

Configuration of GitLab

In gitlab.yml (presumably in /home/git/gitlab/config/ if you installed from source), I have the following:

production: &base
    ## Web server settings (note: host is the FQDN, do not include http://)
    host: gitlab.domain.tld
    port: 443
    https: true

And yes I indicate https true and port 445 even if Gitlab itself does not handle the https part of the communication.

Configuration of Nginx

In the config of Nginx (another layer of reverse proxy), I have:

    proxy_set_header    X-Forwarded-Proto   https;

which differs from the configuration proposed by the Gitlab team. Indeed, it does not convey the protocol that is seen by Nginx but rather forces the header to https as initially the communication was https-secured.

And it works flawlessly!

Node et socket.io derrière Apache en Reverse Proxy

Par PAB le lundi 18 juin 2018, 12:03

Je viens de déployer une application NodeJS basée sur Angular et utilisant socket.io pour communiquer entre le "front-end" et le "back-end". Dans Firefox, je voyais toutefois ce message d'erreur :

Firefox can’t establish a connection to the server at wss://my.app/socket.io/?token=verylongtoken&EIO=3&transport=websocket&sid=verylongsid

et pourtant l'application fonctionnait parfaitement.

Après enquête, il semble que socket.io avait le bon goût de faire un "fallback" sur une méthode AJAX et s'accomodait donc du non établissement du WebSocket.

Pour régler proprement le problème, il fallait :

  • activer le module proxy_wstunnel d'Apache
a2enmod proxy_wstunnel
service apache2 restart
  • puis ajouter ces lignes dans la configuration (attention, le module rewrite d'Apache doit aussi être chargé !)
  RewriteEngine On
  RewriteCond %{REQUEST_URI}  ^/socket.io            [NC]
  RewriteCond %{QUERY_STRING} transport=websocket    [NC]
  RewriteRule /(.*)           ws://localhost:3000/$1 [P,L]

Et hop, tout était fonctionnel désormais !

dimanche 20 mai 2018

One Plus X E1001 and LineageOS

Par PAB le dimanche 20 mai 2018, 06:41

Installing LineageOS on a One Plus X E1001 (not E1003, I believe E1001 is an Asian version whereas E1003 was old in Europe) was a bit painful. This article will not repeat all instructions to install LineageOS on One plus X (which can be found here: https://wiki.lineageos.org/devices/onyx/install) but only cover some unexpected behaviors I need to circumvent to install LineageOS.

First no recent (3+) TWRP version was running fine: all refused to boot (fastboot boot path/to/recovery/file.img) with the following error message:

Remote: dtb not found error

I had to use an old TWRP 2.8.7 version to make it work. Here is the version that worked fine for me: twrp-2.8.7.0-6.0-onyx.img.

Then, the LineageOS zip file could not install because of the following error message:

Comparing TZ version TZ.BF.2.0-2.0.0137 to TZ.BF.2.0-2.0.0134
assert failed: oppo.verify_trustzone("TZ.BF.2.0-2.0.0137") == "1"
E:Installation aborted.

To fix this one, I had to first flash the firmware/bootloader with a more recent version. Installing the following file with TWRP did the trick: OPXBLUpdateOOS3.1.zip.

After that, LineageOS (one of the nightly version of May 2018) installed just fine.

I hope this will help some people facing the "dtb not found" and "assert failed: oppo.verify_trustzone" error messages!

samedi 12 mai 2018

HAProxy et compression Gzip

Par PAB le samedi 12 mai 2018, 18:00

Pour accélérer l'accès à vos sites et services web propulsées par HAProxy, il est facile d'activer la compression automatique des contenus texte comme HTML, CSS, JS (s'ils ne le sont pas déjà par le serveur sous-jacent).

Rien de plus simple, il suffit d'ajouter les 2 lignes suivantes dans la section defaults de la configuration d'HAProxy :

compression algo gzip
compression type text/html text/plain text/xml text/css text/javascript application/javascript application/json text/json

et redémarrer HAProxy par une commande comme

/etc/init.d/haproxy restart

" Avec cette fonctionnalité activée, vous devez désormais voir dans les en-têtes des transmissions entre le serveur web et le navigateur que le contenu est compressé avec gzip (vérification facile avec le module "Développeur" de Firefox par exemple, onglet "Réseau") :

Content-Encoding: gzip

dimanche 6 mai 2018

HAProxy in front of Unicorn

Par PAB le dimanche 6 mai 2018, 07:19

Unicorn, the "Rack HTTP server for fast clients", is really for "fast clients". Fast clients mean clients on the same local network (or even same machine) with very low latency and near-zero chance of packets loss. Slow clients are the usual web users: far away from the server. Unicorn has been designed to serve fast clients and if you try to use it to serve slow clients then the performance may be dismal. This is why the Unicorn developers recommend to use Nginx in front of Unicorn. To know more on the Unicorn philosophy, visit this page.

I recently tried to use HAProxy in front of Unicorn and was disappointed to see that:

  • the system was slow and unresponsive
  • a lot of 502 Gateway errors popped up for seemingly no reason (and this popped up unconsistently)

I came to the conclusion that the default configuration of HAProxy was not appropriate for Unicorn. After some web digging, I discovered the "http-buffer-request" option.

Here is what the HAProxy 1.8 documentation says about the "http-buffer-request" option :

It is sometimes desirable to wait for the body of an HTTP request before taking a decision. This is what is being done by "balance url_param" for example. The first use case is to buffer requests from slow clients before connecting to the server. Another use case consists in taking the routing decision based on the request body's contents. This option placed in a frontend or backend forces the HTTP processing to wait until either the wholebody is received, or the request buffer is full, or the first chunk is complete in case of chunked encoding. It can have undesired side effects with some applications abusing HTTP by expecting unbuffered transmissions between the frontend and the backend, so this should definitely not be used by default.

It seems to be the best fit with Unicorn's philosophy! Let's activate the option in each backend corresponding to a Unicorn-run application:

backend unicorn-app
	option http-buffer-request
	server unicorn-app 1.2.3.4:3000

and restart HAProxy:

/etc/init.d/haproxy reload

- page 1 de 14