Loading time in development

I’ve seen web applications taking about 1 minute to load in the browser while developing. After you hit refresh, you just wait… 40, 50, 60 seconds.

Same for background services: you write code, then “sudo service name restart” and wait 20, 30 seconds to have the service up again.

It’s very sad there are developers who are OK with that, they don’t complain, they just do their work and… wait. And some are doing this for years.

I try to understand each unpleasant situation I meet, some of them having deep roots that are difficult to break (not impossible). Although, the case of such poor loading time in development overwhelms my patience. I find no reason this should be accepted for. It hits the product, the business itself.

Avoiding IF statements for business rules

One moment a project can get on an unwanted road is when new business rules pop in. Even more if this happens on a tight deadline. You just throw some innocent if statements, and that’s how hell can unleash…

Later, other rules will be requested by business. Each will have at least one if statement and maybe interactions with some services. Then one day you notice yourself tangled in the code you’d prefer to never see.

There are ways to implement these rules fast and efficient, avoiding most of those undesired if statements. This is my situation:

  • I have a Song resource and a User resource
  • A Song can be restricted to a User based on multiple conditions
  • If a Song is restricted to a User, we have to let them know why, and this can be for reasons like:
    • Song is not released yet
    • Song was retired by the record company
    • User is from a country where we don’t have rights to offer the Song
    • User has a plan which doesn’t allow him to play the Song
    • and so on

The “what” doesn’t matter, the “how” is the key. And business folks can change their mind any time. They want a new rule fast, or cancel one, or change it. Continue reading Avoiding IF statements for business rules

Types comparison and functions

PHP will let you change a variable’s type at any moment, and there’s nothing you can do about it. While you can use data structures, argument type declaration, and return type declaration to save a lot of damage, you can still change types.

<?php

declare(strict_types=1);

function showTypes(int $number) {
    echo gettype($number) . "\n"; //integer

    $number = "string";
    echo gettype($number) . "\n"; //string
}

showTypes(2);

In order to help this situation a little bit, I’m using the appropriate type comparison operators and functions:

<?php

$number = 1;

// OK
if ($number > 0) {}

// Not OK
if (!$number) {}
if ($number === '') {}
if (strlen($number) > 0) {}


$string = 'Lorem ipsum';

// OK
if ($string === '') {}
if (strlen($string) > 0) {}

// Not OK
if (!$string) {}


$list = [];

// OK
if (count($list) > 0) {}

// Not OK
if (!$list) {}


$bool = false;

// OK
if (true === $bool) {}
if (!$bool) {}

// Not OK
if ($bool == '') {}


$var = null; // if I know a var can be null

// OK
if (null === $var) {}
if (is_null($var)) {}

// Not OK
if (!$var) {}

As such, I’m being more specific about the type variables are holding. In clean code it’s faster to see what a variable holds, but in legacy systems you can see a variable hundreds of lines later after declaration, or you can get a value from a function with no return type or DocBlock.

Treat each variable as close as possible to its type.

Using data structures for types

Data types should be very specific. Anyone using a variable should know exactly what type it is, how it looks like (if it’s a structure). While for some languages it’s common sense and really enforced, others will let you mess with a variable’s type, no matter it’s a primitive, an object, an array.

I’m going to talk about data coming from JSON, databases or other sources, which can be represented into data structures.

In a language like Go, you map data into well defined structures.

package main

import (
    "encoding/json"
    "fmt"
)

type Person struct {
    ID int
    Age int
    Name string
}

func main() {
    string := `{
        "id":453,
        "age":26,
        "name":"John Doe"
    }`

    input := []byte(string)

    person := Person{}
    err := json.Unmarshal(input, &person)
    if err != nil {
        panic(err)
    }

    fmt.Println(person) // Person struct
    fmt.Println(person.ID) // integer
    fmt.Println(person.Age) // integer
    fmt.Println(person.Name) // string
}

You know exactly that you have an object of type Person, with integer ID, integer Age, and string Name. No need to check anything anywhere. If you mess up, you’ll know at compile time.

A dynamic typed language like PHP has a different approach. Continue reading Using data structures for types

Compile Libvirt for Python on Ubuntu

Installing Libvirt on Ubuntu for Python is as easy as:

sudo apt install -y python python-pip libvirt-dev
pip install libvirt-python

 

With the default installation, you could miss some of the Libvirt API bindings exposed to the Python package, although you have the latest version.  So if you ever need to compile the library yourself, here you go (my setup was Libvirt 4.0.0 on Ubuntu 16.04 with Python 2.7):

#!/usr/bin/env bash

WORK_DIR="/tmp/libvirt"

sudo apt update
sudo apt install -y git

# LIBVIRT

WORK_DIR_LIBVIRT="$WORK_DIR/libvirt"
mkdir -p $WORK_DIR_LIBVIRT
cd $WORK_DIR_LIBVIRT

LIBVIRT_VERSION="v4.0.0"
git clone -b $LIBVIRT_VERSION --single-branch --depth 1 https://github.com/libvirt/libvirt.git .
git checkout $LIBVIRT_VERSION

sudo apt install -y \
	gettext \
	libtool \
	autoconf \
	autopoint \
	pkg-config \
	xsltproc \
	libxml2-utils
./bootstrap

sudo apt install -y \
	libnl-3-dev \
	libnl-route-3-dev \
	libxml2-dev \
	libdevmapper-dev \
	libpciaccess-dev \
	python
./configure

sudo apt install -y intltool
aclocal

make
sudo make install


# LIBVIRT PYTHON

WORK_DIR_LIBVIRT_PYTHON="$WORK_DIR/python"
mkdir -p $WORK_DIR_LIBVIRT_PYTHON
cd $WORK_DIR_LIBVIRT_PYTHON

LIBVIRT_PYTHON_VERSION="v4.0.0"
git clone -b $LIBVIRT_PYTHON_VERSION --single-branch --depth 1 https://github.com/libvirt/libvirt-python .
git checkout $LIBVIRT_PYTHON_VERSION

sudo apt install -y python-dev

python setup.py build
python setup.py install


# CLEANUP

rm -r $WORK_DIR
sudo apt purge -y \
	gettext \
	libtool \
	autoconf \
	autopoint \
	pkg-config \
	xsltproc \
	libxml2-utils \
	libnl-3-dev \
	libnl-route-3-dev \
	libxml2-dev \
	libdevmapper-dev \
	libpciaccess-dev \
	intltool \
	python-dev

The above may vary depending on your Ubuntu setup. Just pay attention to errors regarding missing tools.

Now check for the symbols you needed:

sudo apt install -y binutils
nm -g /usr/local/lib/libvirt.so

 

I recommend compilation attempts inside an isolated environment.

Resources

Along the years I’ve read many articles, tutorials, and some books (I’ve watched some videos, too, but I prefer reading). Also, I’ve took a look on some tools to get new ideas. It really helped me in my carrier.

Before doing something, you should at least have an idea of how to do it, if not a plan, and this can come from studying.

Make a little effort to read something (even source code) at least a few times a week.

Planning to read someday: Designing Data-Intensive Applications

Isolation

Big fan of isolation here, from code libraries to apps environments.

Code should be organized in reusable and extensible units (package, library, component), isolated from other units, the interaction between them being made on APIs well described by a contract. Also, they should be easily extensible.

Apps should live in isolated environments. I’m talking about servers, virtual machines, and containers. If different services run on the same machine, you can isolate them using containers. As such, you can safely and independently deploy, make upgrades, balance traffic, move over different machines.
If you must work directly in the container (maybe perform some upgrade), and something goes wrong, you just restart the container from the original image.

I remember having some issues upgrading a Python package in an old production environment. The attempted upgrades just crashed. I knew the service could stay offline for a while, but I had to put it back up. As other services were running on that machine, I didn’t want to interfere with them. So I just set up a new environment for the Python app inside a Docker container, installed it from scratch, and it was up again.

Isolation also suites well legacy apps. You can just throw everything inside a container, and never be afraid of moving between machines, interfering with other services, or breaking things.

Fast on the market and software architecture

You want to be ASAP on the market. So you require developers to write code fast.

We should have a decent code at least, so we can overcome future situations.

Yeah, yeah, but not now, let’s release fast. Then we’ll come back, this is just a concept, we need our clients to see something/we need to fix the issue now (cause it’s burning now, we’ve been ignoring it for a while because it was never urgent).

Both are right. You need good code and fast written code. Can you have them both? I would say not really.

Instead, you can have a decent architecture. At least decent. Start your project, feature, class or function in a manner which will allow you to update code in the moment you realize things went wrong, or when you just need to extend.

You don’t understand, you’re not mature enough, we have to work fast.

Great, I look forward for you to ask me why development is taking so much, why we have bugs.

Use frameworks and/or patterns/advices that others are using and working on for years. If at some point you end up on an unpleasant road, it’s going to be easier to change. Don’t be afraid of things you’re not used to.

And try to work as isolate as possible. Build independent components/libraries/modules/packages that are easy to integrate and dispose when needed, do not throw code everywhere.

Do not try to guess the future, make sure the present allows you to adapt. Otherwise, time, effort, money will be invested in maintaining a monster which holds you back from being fast on the market.

If it’s not broken, fix it before it will be

I’ve recently read a discussion on server software upgrades, and someone was recommending not to upgrade if you don’t have a problem.

My approach is a little bit different, as in I advise you to upgrade from time to time (at least) your OS and packages. In “the real world” it’s not often easy to do this, but if context allows, you should do it. You should prevent unwanted situations, instead of fixing them after they’ve done damage, and maybe suffer consequences.

A basic approach (in a decent context) would be to make upgrades in test environments. After you make sure you’re ready for production upgrade, redirect traffic to some new temporary machines before updating main machines. Of course, if you have your own load balancer instead of a DNS one, it’s easier to do this. Don’t upgrade directly on production, don’t assume nothing could go wrong. Back up data before!

The same goes for code libraries. I’ve seen upgrades being ignored because “we don’t have time now”, and ignorance. Then a moment came when they were forced to make the upgrades, because new libraries were depending on new versions of the existing ones. You should have seen the “joy”, and the bugs which followed in production, as there was no time to test everything as it should have been.

Upgrades can be a pain, they can crash, they can take time, new packages can have issues. However, this is not a reason to ignore them.

Hello, interviewe(r/e)!

An interview should be in both directions. Both the interviewer and the interviewee should have questions to ask, as you know, of course.

While the interviewer is expected to have any kind of questions and requests, I’ve seen most interviewees lack some things I always need from a company in order to get to know them better.

I’ve been on both sides, and I really consider that if you go to an interview, you should ask about the following, and I’m really happy when I’m asked about them:

  • Code. There is no reason for a company to refuse to show you code. 100% there is something that can be shown.
  • Frameworks. If they use custom frameworks, ask details on how they manage some basic situations: models, repositories, dependency injection, dependency manager/packages, events, and so on. And why did they need a custom framework?
  • Tests, build, deploy, devops.
  • Databases. Structure can tell you some things about the project.
  • Team. Is there anyone you will learn from? How many members are there in the team?
  • Loading/compile time in development. If it’s a browser app, ask to see a page refresh. If it’s a cli app (even more if it’s a long live app), ask to see how they reload.
  • Code versioning system.
  • Workflow. From specifications to release. How do they manage the whole process?
  • Critical situations management. What happens if something doesn’t go as planned: bugs in productions, servers crashes, people issues.
  • Hardware and software. Your working environment (desktop/laptop/ide) should be with you, not against you.

I’m not saying you should decline a company if they don’t satisfy you regarding all of the above. Just know them better and make the best decision for you.