This post assumes the reader knows how to use the MySQL command line and has access to the servers they are trying to set up replication between. It also assumes that network access on the master has been configured correctly (including a user with replication permissions on the master).

Foreword

While this post contains nothing about MySQL replication that’s exclusive to setting it up on a Raspberry Pi. I’m just using this category of the blog to archive what I’ve managed to make my Raspberry Pi do.

Setting up replication on the master server

Configuring the master server to support replication is an easy task. All that is needed is either to enable binlog for the entire server. Make sure that a server ID for the master is set as well (server-id=1 or similar in the config).

log-bin=mysql-bin

It’s also possible to enable replication for specific databases only:

binlog_do_db=wordpress

Keep in mind that using binlog_do_db can lead to inconsistencies because it looks at what database is in use and not what database a query acted upon. Thus, enabling binlog for the entire server is the recommended approach.

Getting data from the master

Getting data from the master is easy, simply flush all tables and lock the database, after that get the status of the master.

FLUSH TABLES WITH READ LOCK;
SHOW MASTER STATUS;

Note the output of this command somewhere and then proceed with dumping the data. Which should display something along the lines of:

+------------------+----------+--------------+------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+------------------+----------+--------------+------------------+
| mysql-bin.000544 |  1473773 |              |                  |
+------------------+----------+--------------+------------------+

Dumping  the database can be done using mysqldump. With something like this:

mysqldump --add-drop-database --add-drop-table --hex-blob --databases <list of databases goes here> | gzip -c --best >database-$(date -I).sql.gz

When the database has been backed up properly the table locks can be released and you can start transferring the  backup to the slave server.

If the database is too large to import (as it was in my case) moving the data files over directly is another option (situated in /var/lib/mysql for Debian servers). Check the position of the log using the command above, shut down the server and move the files over. Then use “CHANGE MASTER TO” from the section below (after setting up a server ID of course).

Unlocking the tables can be done using:

UNLOCK TABLES;

Transfer the database dump to the slave server using your tool of choice (e.g. scp).

Configuring the slave server

After moving the database dump over to the slave we can get started with configuring the slave server.

Importing the data into mysql on the slave server is easy, it’s simply a matter of using gzip and piping it into mysql (make sure there’s not too much data for it to handle):

gzip -c -d database-2014-03-05.sql.gz | mysql -u root -p

If you transferred the data over in the previous step using scp or similar then extract it into /var/lib/mysql (unless skipped archiving and simply moved it there directly, which is fine).

After this, shut down the MySQL server and add a server ID to the slave. If importing data files directly then shut down the mysql server before importing.

# And set up a server ID (any 32-bit integer should do, as long as it doesn't match the master)
server-id=70078

If you want to replicate only some databases that can be done using “replicate-do-db” although it has the same problem as binlog_do_db has (i.e. might introduce inconsistencies).

If you wish to have the host name of the slave appear when checking the status of the slave servers on master then add this as well:

report-host=hostname-here

Now, start the MySQL server again and run the following command in the mysql command prompt (note: it’s using the information from the SHOW MASTER STATUS above):

CHANGE MASTER TO
  MASTER_HOST='master.address.here',
  MASTER_USER='replicationuser',
  MASTER_PASSWORD='secritpassword',
  MASTER_PORT=3306,
  MASTER_LOG_FILE='mysql-bin.000544',
  MASTER_LOG_POS=1473773,
  MASTER_CONNECT_RETRY=10;

And then run:

START SLAVE;

Everything should be up and running, you can check the status of the slave server by running the following on the slave:

SHOW SLAVE STATUS;

Errors such as connection failures and similar are listed in the output there. Finally, checking the slave status on the master can be done using these:

SHOW SLAVE HOSTS;
SHOW PROCESSLIST;

And there you have it, MySQL replication set up on a RPi. :3

Prerequisites: This blog post assumes the reader knows how to set up a zone using bind as well as the general syntax of bind’s configuration files.

Background

Recently, I got a Raspberry Pi to play around with and I decided to set up a secondary DNS server. While I have no real need for a secondary DNS server I figured it would be a good exercise to set it up. It also allows me to bring down the server for an upgrade without losing name resolution on the network so there’s really no reason not to do it.

To start off I installed Bind9 using

pacman -S bind

Configuring bind

Setting up the Pi as a secondary DNS server was easy.  It was just a matter of adding A-, AAAA-, and NS-records for it and then changing the configuration files of the master and slave to allow transfers between the two.

Which ended up being added as follows (to the chalamius.se zone):

chalamius.se. 3600 IN NS luna.chalamius.se.
luna.chalamius.se. 3600 IN A 192.168.x.x
luna.chalamius.se. 3600 IN AAAA

And a number of NS-records for the other zones (DHCP hosts and reverse DNS).

Next, configuring zone transfers was as simple as adding the host to the ACL for transfers I have set up (but it works just as well with addresses).

To add a host to allow-transfer one simply adds the following (between the omitted parts, […] signifies omitted parts of the config) to the zone-declaration in the master’s config file:

zone "chalamius.se." IN {
    type master;
    [...]
    allow-transfer {
        [...]
        192.168.x.x;
    }
    [...]
}

Configuring the Pi as a slave is simply a matter of adding a slave zone-declaration to the config file (one for each zone that it’s supposed to handle), which is done as follows:

zone "chalamius.se." IN {
    type slave;
    file "/var/lib/named/chalamius.se.hosts";
    masters {
        192.168.x.y;
    };
    allow-transfer { none; };
};

Verification

Verifying that it works is simple, issue a name resolution request and see if the response is positive. To verify that transfers are working one simply adds a host to one of the zones that are configured for transfers, update the serial and reload the zones on the master.

Verifying that transfers work without doing that (assuming bind doesn’t do it automatically upon reloading the configuration file, which it should) can be triggered using (replace with your zone of choice):

dig axfr chalamius.se

Verifying that name resolution works can be done by using:

dig +norecurse @192.168.x.x newhost.chalamius.se

Next up

Assuming I manage to find the energy to write something on this  blog sometime soon I’ll put up a post on how to configure MySQL replication using a Raspberry Pi.

Some facts about Southrop:

He

• is a staff member over at BakaBT
• runs Chaos, an anime fansub group (among other things)

Well, that’s all. Head over to Chaos|Theory and congratulate him already!

The quote database I work on (Akyu) have been updated to 0.2 recently (24th August).
This changed some things:

• Voting buttons now actually tells you they are, i.e voting buttons
• No text error messages, all error messages are displayed when you hover over the button you just clicked
• Some minor changes to the layouts of some pages (for example the search page)

pIRC

A smaller bug fix a couple of days ago. The event handler couldn’t handle some constants as triggers.
Away has been added to the irc wrapper as well.

Going to try to get some sort of Admin module into it next time I have a longer weekend than I have now.

Voile

My little image gallery I work on every now and then.
It currently only generates cached thumbnails for what I upload and lists them on the index page; nothing fancy.

Feel free to take a look

One last thing…

Mandatory Touhou

Tenshi, that is all. (Anyone thought of Angel Beats! Tenshi? No? Good.)

Either way, recently I realized how nice Tenshi’s theme is. You should check it out (a simple search for “Catastrophe in Bhava-agra ~ Wonderful Heaven” on YouTube will give you plenty results or her final spell theme for that matter “Bhava-agra As Seen Through a Child’s Mind”).

Also, Git.

I switched most of my projects over from SVN to Git, which is quite a lot better IMO. Not to mention it’s quite a bit faster and works without an internet connection (which is a plus if you’re on the bus or similar).

I’m yet to experiment with the more hidden features of Git though but I’ll get to that soon enough. So far so good though, Git is good.

Anyways

I have a rewrite of both the quote database and Lolikins going on (Lolikins is being moved over to the new IRC Lib), the QDB is being rewritten to be easier to manage.

The Quote Database’s appearance probably won’t change at all considering I’m the lazy person I am.
Lolikins won’t have anything major changed except she replies to a few more CTCP commands (kind of got exchange rates planned for her though, but that’s up to Mirgond to help me figure out some good exchange rates).

And finally

That’s all. I just thought I’d write something again to not let this blog die out.

tl;dr I’m bored.

As you might know (at least if you checked out the Projects tab) I’m working on an irclib, so I thought I’d give you a bit of a preview on how the actual ircbot looks from the command-line.

Sadly it seems like the coloring and stuff will be Linux only since it requires an extra (3rd-party, not Microsoft) executable for Windows.

An extensible IRC-bot made in PHP using as few PECL modules as possible.

The goal of pIRC is to have a fast lightweight bot that’s extensible.

And this is the current feature set:

– Event Handler

• Supports both arrays and single arguments, if none are given upon registering the function will be passed the standard Message object.
• Built-in flood protection

– Modules

• All modules can be loaded, unloaded and reloaded during run-time
• It checks the syntax before loading to prevent accidental crashing
• Probably less leaky than the rename class method that others seems to use

Anyways,

I did some updates to the QDB to make it a bit faster along with fixing up a new theme for my #bakabt stats.

Sadly not much more to report, except the fact that I have math test on Wednesday so I wont get anything done on anything this week anyway :3

Also, if you have any suggestions to the QDB or similar, feel free, suggestions are always welcome.

This post is pretty much a post to remind myself to work on this crap when I get the time to >_>
also, todo: Add more stuff to this update post.

Lambda Expressions were introduced in the 3.0 version of the Common Language Specification.  Lambdas are essentially a refinement of anonymous methods.  They allow you to define a block of code which operates on one or more input variables and returns a single result.  Let’s break down a simple lambda example, from the MSDN page on Lambda Expressions:

delegate int del(int i);
static void Main(string[] args)
{
    del myDelegate = x => x * x;
    int j = myDelegate(5); //j = 25
}

In this example, a delegate type is defined, del.  Then an instance of the delegate is created using a lambda expression.  Finally, the delegate is executed, and the return value is assigned to the integer variable j.  Let’s ignore everything but the lambda expression itself: x => x * x.

The first character of the expression actually defines a variable named x, which only exists inside the lambda expression.  In the MSDN example, when the lambda is actually executed, x will be assigned the input value of 5.

The => character sequence tells C# that your lambda expression operates on the variable x.  To oversimplify, everything to the left of => is an input variable declaration, and everything on the right is an expression which determines the return value.  In our examples, there will only ever be one input variable.

The x * x character sequence is the actual expression, the bit of code that gets executed.  So we could rewrite down the code above to this:

int x = 5;
int j = x * x;

…and achieve the same result.

Filtering Cars with Lambda Expressions

Returning to our anonymous methods example, we found that the FindAll method of the List<> generic class can take a delegate. The delegate is used to filter which items in the list are returned.

With Anonymous Methods

static void Main(string[] args)
{
    // Create a list of cars meeting criteria by calling the FindAll()
    // method of the List returned by CreateCars().
    List cars = CreateCars().FindAll(delegate(Car candidate)
    {
        // Return whether the car matches our criteria.  FindAll()
        // expects our anonymous method to return a boolean, and
        // the compiler makes sure this is the case.
        return
            candidate.Manufacturer == "Toyota" ||
            candidate.Manufacturer == "Nissan" &&
            candidate.Turbocharged == false;
    });

    foreach(Car car in cars)
    {
        Console.WriteLine(car.ToString());
    }
}

Instead, let’s replace that anonymous method block with a lambda expression.

With Lambda Expressions

static void Main(string[] args)
{
    List cars = CreateCars().FindAll(car =>
        car.Manufacturer == "Toyota" ||
        car.Manufacturer == "Nissan" &&
        car.Turbocharged == false );

    car.Turbocharged = true; // Invalid code, "car" does not exist.
}

In this case, our lambda expression creates the variable car, and then does the same set of conditional tests as the old example. Note that the car variable only exists in the scope of the lambda expression, just like in the anonymous method. For that reason, the second to last line of code would be a compiler error.

So What’s the Difference?

For the purposes of our example, not a whole lot:

• The input variable and return types are inferred. C# is able to figure out what type car is by looking at where the lambda is used.   The type is actually defined in the CreateCars() method of our earlier example. C# looks at the return type of List<Car> and infers (or deduces) that a lambda expression operating on the FindAll method of that List<Car> would operate on a single input variable of type Car. Note that in the old anonymous methods example, C# was only able to infer the return type.
• The lambda expression does not need curly braces, since it is a basic expression.
• The lambda expression does not use the ugly delegate() syntax, thanks to the type inference.

Of course lambda expressions are a bit more complicated than I let on; they can declare multiple input variables, and they can also contain normal procedural code statements (such as calling a method — and that does require using curly braces).

Go to his blog and congratulate him, if he posts a post about his birthday that is :3

Now go congratulate him.

On another note I’m working a bit on the QDB and on Lolikins, again.
You can expect an update on one of them like… next month or something, blame school for making my projects appear stalled ;_;