Author: Andrei

GCC bug in noexcept operator

When something goes wrong, the first thing is to ask myself what did I do wrong. I don’t like just throwing the blame. And when I saw the not declared in this scope error related to the noexcept operator,  I was sure I was missing something. Maybe I had accidentally used a compiler extension, maybe I used something that’s undefined behavior. But I didn’t think it could be a GCC noexcept compiler bug.

I have a container that encapsulates a vector, for which I wanted to have a noexcept swap method. Ignore the implementation itself, the idea is I wanted to declare the method noexcept if the swap operations on the items I wanted to swap are also noexcept. It can be any other case that implies noexcept.

#include <vector>
#include <utility>
#include <cassert>

struct Container {
    std::vector<int> elements{};
    std::size_t count{};

    void swap(Container &c) noexcept(noexcept(elements.swap(c.elements)) && noexcept(std::swap(count, c.count))) {
        elements.swap(c.elements);
        std::swap(count, c.count);
    }
};

int main() {
    Container a{{1, 2}, 2};
    Container b{{3, 4, 5}, 3};

    a.swap(b);

    assert(a.elements == (std::vector<int>{3, 4, 5}));
    assert(a.count == 3);

    assert(b.elements == (std::vector<int>{1, 2}));
    assert(b.count == 2);
}

 

This tells that Container’s swap is noexcept if both swaps of elements and count are noexcept:

void swap(Container &c) noexcept(noexcept(elements.swap(c.elements)) && noexcept(std::swap(count, c.count)));

 

MSVC and Clang compile this, but GCC needs a newer version because on older ones it has the bug [DR 1207] “this” not being allowed in noexcept clauses, which I’ve found in this discussion.

If you see one of the following errors, try to update your compiler:

    • ‘elements’ was not declared in this scope
    • invalid use of incomplete type ‘struct Container’
    • invalid use of ‘this’ at top level

String processing: Mask an email address

While studying STD algorithms in C++, one simple exercise I did was masking an email address. Turning johndoe@emailprovider.tld into j*****e@emailprovider.tld, considering various cases like very short emails and incorrect ones (one could impose a precondition on the input, that it must be a valid email address to provide a valid output, but for this exercise, I wanted some edge cases).

To know what kinds of inputs I’m dealing with and what the corresponding valid outputs should be, I’ll start with the test data:

const std::map<std::string, std::string> tests{
        {"johndoe@emailprovider.tld", "j*****e@emailprovider.tld"},
        {"jde@emailprovider.tld",     "j*e@emailprovider.tld"},
        {"jd@emailprovider.tld",      "**@emailprovider.tld"},
        {"j@emailprovider.tld",       "*@emailprovider.tld"},
        {"@emailprovider.tld",        "@emailprovider.tld"},
        {"wrong",                     "w***g"},
        {"wro",                       "w*o"},
        {"wr",                        "**"},
        {"w",                         "*"},
        {"",                          ""},
        {"@",                         "@"},
};

Besides solving the task itself, I was also curious about an aspect: What would be the differences between an implementation using no STD algorithms and one using various STD algorithms? I followed how the code looks and how it performs.

The first approach was the classic one, using a single iteration of the input string, during which each character is checked to see if it should be copied to the output as is or it should be masked. After the iteration, if the character @ was not found, the propper transformation is done.

std::string mask(const std::string &email, const char mask) {
    if (email[0] == '@') {
        return email;
    }

    std::string masked;
    masked.reserve(email.size());

    bool hide = true;
    bool is_email = false;

    for (size_t i = 0; i < email.size(); ++i) {
        if (email[i] == '@') {
            is_email = true;
            hide = false;
            
            if (i > 2) {
                masked[0] = email[0];
                masked[i - 1] = email[i - 1];
            }
        }

        masked += hide ? mask : email[i];
    }

    if (!is_email && masked.size() > 2) {
        masked[0] = email[0];
        masked[masked.size() - 1] = email[masked.size() - 1];
    }

    return masked;
}

Continue reading String processing: Mask an email address

The C++ Programming Language (4th Edition)

by Bjarne Stroustrup

About a week ago I finished reading The C++ Programming Language (4th Edition), a book on C++11. It enlightened me in some ways, by understanding how and why some things are done, and I got to know about a big part of the language.

Why did I read about an almost 10 years old C++ standard? I didn’t know where to start and I didn’t want to lose too much time thinking about the best way to learn the language. I wanted to know about the language and start writing code; this is what works for me. In the past, with other languages, I started by writing code and left reading for a later time, but I wasn’t that happy with the result. Moreover, I knew who the author of the book is, I trusted him, so I just started reading.

The 2011 standard is still relevant today as it brought major changes to the language. On the other hand, newer standards brought fewer changes, but very important. But this was going to be the next step after reading the book, which I also did. I got up to date with C++14 and 17 (I’ve seen things about C++20, but I didn’t want to get into details yet).

What did I gain? The most important gain is understanding some aspects of how the language works. As I write code, I remember some things I should pay attention to, some techniques that could help me, some ideas and keywords that I should be looking for, and where to go next. Now I know what the language offers, even if there are a lot of parts and important details that I don’t remember or understand.

How was my reading process? It was a lot about writing. I didn’t just read the book like a story. I actually wrote almost all of the code that was presented to get used to the syntax, to fill in where parts were missing, to practice. After every chapter, I wrote the code, changed it, ran the debugger, ran Valgrind to see if I have leaks. And all along I implemented various little things like queues, stacks, and other exercises.

Do I know C++? Of course not. I know ABOUT C++ and some of its components that can help me write better code. Only practical experience actually teaches me. While I write code, I have many hints in my mind about how to use the language.

I did this once before with Go and it really, really helped me to know my tool. It opened my eyes and my mind. It’s something I strongly recommend. Get your hands dirty with code while knowing what you got your hands on.

Trim std::string implementation in C++

I was working with some strings and I wondered how you can trim a string in C++. Having iterators and so many algorithms (too many?) in the Standard Library gives a lot of flexibility, and some tasks were left out of the standards.

The flexibility of C++ feels like morally pushing you to also write flexible code which can cover a lot of needs. Most probably some things could be improved to my implementation of string trimming.

The functions are ltrim (erase from left), rtrim (erase from right) and trim (erase from left and right). All three take a reference to a string (the input string is modified) and a predicate function to match the characters you want to erase (std::isspace as default):

using Predicate = std::function<int(int)>;

static inline void ltrim(std::string &str, Predicate const &pred = isspace);

static inline void rtrim(std::string &str, Predicate const &pred = isspace);

static inline void trim(std::string &str, Predicate const &pred = isspace);

Continue reading Trim std::string implementation in C++

Pass function to thread (job scheduler update)

A simplification to the job scheduler from the previous post is to pass the job function to the thread managing the job call, instead of making a shared pointer and capture it in the lambda.

The schedule method changes to:

void Scheduler::schedule(const Job f, long n) {
    std::unique_lock<std::mutex> lock(this->mutex);
    condition.wait(lock, [this] { return this->count < this->size; });
    count++;

    std::thread thread{
            [this](const Job f, long n) {
                std::this_thread::sleep_for(std::chrono::milliseconds(n));

                try {
                    (*f)();
                } catch (const std::exception &e) {
                    this->error(e);
                } catch (...) {
                    this->error(std::runtime_error("Unknown error"));
                }

                condition.notify_one();
                count--;
            }, f, n
    };
    thread.detach();
}

A job scheduler in C++

Not long ago I started writing some C++ code, and a task that I enjoyed implementing was a very basic job scheduler (idea from dailycodingproblem.com). I’m sure there are “holes” to be filled in my implementation regarding performance, concurrency, and general correctness. This is an early dive into the language and this post is mostly for me, to explain some things to myself.

The scheduler takes a job (function) and a time (in milliseconds) and runs the job after the given time.

The first thing was to define a function pointer as the job type.

using Job = void (*)();

 

Another callback that I’ll be using is one to report job errors, which takes an exception as an argument.

using Error = void (*)(const std::exception &);

 

The scheduler class constructor takes a size and an error callback (to report errors). The size is the maximum number of jobs accepted until scheduling blocks and waits for a job to be finished.
An error callback is required. The first measure to ensure this is to delete the constructor that takes null for the error callback, which performs a compile-time check. Explicitly passing null will not be allowed, but a pointer that is null will be checked at runtime.

Scheduler(size_t size, Error error);
Scheduler(size_t size, nullptr_t) = delete;

Continue reading A job scheduler in C++

Docker and iptables

The firewall must be configured properly to prevent unwanted access which could lead to data loss and exploits. UFW allows you to quickly close and open ports. The following configuration closes all incoming ports and opens 22 (SSH), 80 (HTTP) and 443 (HTTPS):

ufw default deny incoming
ufw allow OpenSSH
ufw allow http
ufw allow https
ufw enable

But Docker will update iptables when you bind a container port to the host, opening the port for public access. To prevent this, you could bind the port to an internal address (private or 127.0.0.1). Another way is telling Docker to never update iptables by setting the “iptables” option to “false” in /etc/docker/daemon.json. This file should contain a JSON string: { “iptables”: false }.

This can be automated as:

apt install -y ufw
ufw default deny incoming
ufw allow OpenSSH
ufw allow http
ufw allow https
ufw --force enable # --force prevents interaction

apt install -y jq
touch /etc/docker/daemon.json
[[ -z $(cat /etc/docker/daemon.json) ]] && echo "{}" > /etc/docker/daemon.json
echo $(jq '.iptables=false' /etc/docker/daemon.json) > /etc/docker/daemon.json

But there are cases when preventing Docker from manipulating iptables can be too much and DNS won’t be resolved to some containers. The issue I met was when building a container from the NGINX Alpine image:

---> Running in 71130dd103f3
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
ERROR: http://dl-cdn.alpinelinux.org/alpine/v3.9/main: temporary error (try again later)
WARNING: Ignoring APKINDEX.b89edf6e.tar.gz: No such file or directory

For containers like this one you could use –network host or manually add iptables rules, which probably is not the best idea because they could change in the future. These are the rules I’ve seen Docker add:

iptables -N DOCKER
iptables -N DOCKER-ISOLATION-STAGE-1
iptables -N DOCKER-ISOLATION-STAGE-2
iptables -A FORWARD -j DOCKER-USER
iptables -A FORWARD -j DOCKER-ISOLATION-STAGE-1
iptables -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -o docker0 -j DOCKER
iptables -A FORWARD -i docker0 ! -o docker0 -j ACCEPT
iptables -A FORWARD -i docker0 -o docker0 -j ACCEPT
iptables -A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
iptables -A DOCKER-ISOLATION-STAGE-1 -j RETURN
iptables -A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
iptables -A DOCKER-ISOLATION-STAGE-2 -j RETURN
iptables -A DOCKER-USER -j RETURN
iptables -t nat -A POSTROUTING ! -o docker0 -s 172.17.0.0/16 -j MASQUERADE

I did not specifically need to stop Docker from updating iptables as I was just experimenting, so I just let it do its job. I’m publishing (-p) the ports I need for public access and exposing (- -expose) the private ones.

Go bookmarks

I like reading again and again some resources which make me understand Go better each time. Some of them are worth being saved as bookmarks, which is what I’m doing right here. I wish I had known some of them a few years ago.

Why:

Intro:

Concurrency and memory model:

Channels:

Profiling:

Scheduling, stacks, pointers, memory:

Algorithms, patterns, tools:

People for people:

PHP 7.4 was released today

I was waiting for PHP 7.4 to be released. The most exciting feature for me is Typed Properties. Being released today, I was eager to compile it just to see that feature in action (and I’m also curious about Opcache Preloading).

I compiled it inside a Docker container and wrote a small test file.

Dockerfile:

FROM ubuntu:18.04

ARG VERSION=7.4.0

RUN apt update
RUN apt install -y wget gcc make automake pkgconf libxml2-dev libsqlite3-dev

RUN wget https://www.php.net/distributions/php-$VERSION.tar.gz
RUN tar xzfv php-$VERSION.tar.gz

WORKDIR php-$VERSION

RUN ./configure \
    --with-pdo-mysql \
    --with-mysqli

RUN make -j $(nproc)
#RUN make test # probably fails for now

RUN make install
RUN php -v

Compile PHP:

docker build . -t php740build

Save this as test.php:

<?php declare(strict_types=1);

class Calc
{
    /**
     * @var float[]
     */
    private array $nums;

    /**
     * Calc constructor.
     * @param float $n
     * @param float[] $nums
     */
    public function __construct(float $n, float ...$nums)
    {
        $this->nums = [$n, ...$nums];
    }

    /**
     * @return float
     */
    public function sum(): float
    {
        return array_sum($this->nums);
    }
}

$calc = new Calc(1, 2, 3);

echo $calc->sum();
echo "\n";

Run the test:

docker run --rm -v $PWD/test.php:/test.php -ti php740build php /test.php

It’s probably better to wait a while for some patches after being tested in the wild wild web.

Go Future with Reflection

While exercising Go with various Future implementations, at one moment reflection hit me. Using reflection to inspect types beats the point of strong types, but it never hurts to play.

I want to use any function within a Future and then call it asynchronous and wait for results. I could have a function to create the Future which holds the user function. Not having generics yet, any function could be passed by interface{}, then called by reflection.

So the user function is passed to a function which returns another function that can be called with the user function arguments. The user function will be executed on a new routine.

// Callable is the function to call in order to start running the passed function.
type Callable func(args ...interface{}) *Future

// New returns a Callable.
func New(function interface{}) Callable {

For a sum function, the usage is:

sum := func(a, b int) (int, error) {
   return a + b, nil
}
future := futurereflect.New(sum)(1, 2)

result, err := future.Result()

The Future should allow waiting for the user function to be executed and get its result.

type Future struct {
   functionType  reflect.Type
   functionValue reflect.Value
   args          []interface{}
   wait          chan struct{}
   result        interface{}
   err           error
}

// Wait blocks until function is done.
func (f *Future) Wait() {
   <-f.wait
}

// Result retrieves result and error. It blocks if function is not done.
func (f *Future) Result() (interface{}, error) {
   <-f.wait
   return f.result, f.err
}

The rest is just implementation, but it feels like pure madness. I did it for fun, to experiment, there are cases missing. And a language should not be pushed where it doesn’t belong. Use each language in its style.

Run the code on Go Playground.

Continue reading Go Future with Reflection