Thursday 28 November 2013

how to add a storage-engine: DUMMY

so let's write some code of our own, let's start with a simple storage-engine. This engine is silly, it is the equivalent of the device /dev/null. It simply does nothing but it contains the minimal amout of code needed for the implementation of such an engine.

All examples in this post are tested with MySQL, the steps are similar for MariaDB (the names can change a bit).

all the files

So let's start: I prepared everything so simply download this file.

Extract this file to the storage-folder of the MySQL-source (or the same folder of the MariaDB-source).

After extracting the data I entered this line on a console:
MySQL
august@AMD4:~/workspace/MySQL-Src/mysql-5.5.8$ cmake  -DWITH_DEBUG=1  -DMYSQL_DATADIR=/home/august/MySQL/data  -DCMAKE_INSTALL_PREFIX=/home/august/MySQL/pgm

MariaDB
august@AMD4:~/workspace/MariaDB/mariadb-10.0.4$ cmake  -DCMAKE_BUILD_TYPE=Debug  -DMYSQL_DATADIR=/home/august/MariaDB/data  -DCMAKE_INSTALL_PREFIX=/home/august/MariaDB/pgm

a hint: these lines are valid for my PC. Please look for the configuration of your PC and set the parameters accordingly.

Now I switched back to Eclipse, recompiled the source and this storage engine is included in the server-code:
  • in the workspace: $HOME/workspace/MySQL-Src/mysql-5.5.8/storage/dummy/ha_dummy.so
  • in the installed-version: $HOME/MySQL/pgm/lib/plugin/ha_dummy.so
and that's all folks.


so let's see if it works

You can find all the output presented here in: storage/dummy/Skripte/Tests.sql
you can read it there or continue reading here.

In Eclipse one can start the server via a click on the Run-icon. In the console of Eclipse the output should end with these lines:
131125 15:33:04 [Note] /home/august/workspace/MySQL-Src/mysql-5.5.8/sql/mysqld: ready for connections.
Version: '5.5.8-debug'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  Source distribution

Opening a console and let's see:
august@AMD4:~/MySQL/pgm$ bin/mysql

Welcome to the MySQL monitor.  Commands end with ; or \g.

Your MySQL connection id is 1

Server version: 5.5.8-debug Source distribution



Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved.



Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.



Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

First of all we look for the installed plugins:
mysql> show plugins;
+-----------------------+--------+--------------------+----------------+---------+
| Name                  | Status | Type               | Library        | License |
+-----------------------+--------+--------------------+----------------+---------+
| binlog                | ACTIVE | STORAGE ENGINE     | NULL           | GPL     |
| mysql_native_password | ACTIVE | AUTHENTICATION     | NULL           | GPL     |
| mysql_old_password    | ACTIVE | AUTHENTICATION     | NULL           | GPL     |
| MEMORY                | ACTIVE | STORAGE ENGINE     | NULL           | GPL     |
| MRG_MYISAM            | ACTIVE | STORAGE ENGINE     | NULL           | GPL     |
| CSV                   | ACTIVE | STORAGE ENGINE     | NULL           | GPL     |
| MyISAM                | ACTIVE | STORAGE ENGINE     | NULL           | GPL     |
| PERFORMANCE_SCHEMA    | ACTIVE | STORAGE ENGINE     | NULL           | GPL     |
| InnoDB                | ACTIVE | STORAGE ENGINE     | NULL           | GPL     |
| INNODB_TRX            | ACTIVE | INFORMATION SCHEMA | NULL           | GPL     |
| INNODB_LOCKS          | ACTIVE | INFORMATION SCHEMA | NULL           | GPL     |
| INNODB_LOCK_WAITS     | ACTIVE | INFORMATION SCHEMA | NULL           | GPL     |
| INNODB_CMP            | ACTIVE | INFORMATION SCHEMA | NULL           | GPL     |
| INNODB_CMP_RESET      | ACTIVE | INFORMATION SCHEMA | NULL           | GPL     |
| INNODB_CMPMEM         | ACTIVE | INFORMATION SCHEMA | NULL           | GPL     |
| INNODB_CMPMEM_RESET   | ACTIVE | INFORMATION SCHEMA | NULL           | GPL     |
| partition             | ACTIVE | STORAGE ENGINE     | NULL           | GPL     |
| FCWTEXT               | ACTIVE | STORAGE ENGINE     | ha_fcwtext.so  | GPL     |
| DBF                   | ACTIVE | STORAGE ENGINE     | ha_dbf.so      | GPL     |
| MYC                   | ACTIVE | STORAGE ENGINE     | ha_myc.so      | GPL     |
| MYC2                  | ACTIVE | STORAGE ENGINE     | ha_myc2.so     | GPL     |
| maxtempl              | ACTIVE | STORAGE ENGINE     | ha_maxtempl.so | GPL     |
+-----------------------+--------+--------------------+----------------+---------+
22 rows in set (0,00 sec)

mysql>

then we add our little dummy-engine to the server and check:
mysql> install plugin dummy soname 'ha_dummy.so';
Query OK, 0 rows affected (0,09 sec)

mysql> show plugins;
+-----------------------+--------+--------------------+----------------+---------+
| Name                  | Status | Type               | Library        | License |
+-----------------------+--------+--------------------+----------------+---------+
| binlog                | ACTIVE | STORAGE ENGINE     | NULL           | GPL     |
| mysql_native_password | ACTIVE | AUTHENTICATION     | NULL           | GPL     |
| mysql_old_password    | ACTIVE | AUTHENTICATION     | NULL           | GPL     |
| MEMORY                | ACTIVE | STORAGE ENGINE     | NULL           | GPL     |
| MRG_MYISAM            | ACTIVE | STORAGE ENGINE     | NULL           | GPL     |
| CSV                   | ACTIVE | STORAGE ENGINE     | NULL           | GPL     |
| MyISAM                | ACTIVE | STORAGE ENGINE     | NULL           | GPL     |
| PERFORMANCE_SCHEMA    | ACTIVE | STORAGE ENGINE     | NULL           | GPL     |
| InnoDB                | ACTIVE | STORAGE ENGINE     | NULL           | GPL     |
| INNODB_TRX            | ACTIVE | INFORMATION SCHEMA | NULL           | GPL     |
| INNODB_LOCKS          | ACTIVE | INFORMATION SCHEMA | NULL           | GPL     |
| INNODB_LOCK_WAITS     | ACTIVE | INFORMATION SCHEMA | NULL           | GPL     |
| INNODB_CMP            | ACTIVE | INFORMATION SCHEMA | NULL           | GPL     |
| INNODB_CMP_RESET      | ACTIVE | INFORMATION SCHEMA | NULL           | GPL     |
| INNODB_CMPMEM         | ACTIVE | INFORMATION SCHEMA | NULL           | GPL     |
| INNODB_CMPMEM_RESET   | ACTIVE | INFORMATION SCHEMA | NULL           | GPL     |
| partition             | ACTIVE | STORAGE ENGINE     | NULL           | GPL     |
| FCWTEXT               | ACTIVE | STORAGE ENGINE     | ha_fcwtext.so  | GPL     |
| DBF                   | ACTIVE | STORAGE ENGINE     | ha_dbf.so      | GPL     |
| MYC                   | ACTIVE | STORAGE ENGINE     | ha_myc.so      | GPL     |
| MYC2                  | ACTIVE | STORAGE ENGINE     | ha_myc2.so     | GPL     |
| maxtempl              | ACTIVE | STORAGE ENGINE     | ha_maxtempl.so | GPL     |
| DUMMY                 | ACTIVE | STORAGE ENGINE     | ha_dummy.so    | GPL     |
+-----------------------+--------+--------------------+----------------+---------+
23 rows in set (0,00 sec)

mysql>
OK, it's there (scroll to the end of the list and you will see it - I changed the output to bold).

So let's use our silly engine:
mysql> create database Test;
Query OK, 1 row affected (0,01 sec)

mysql> use Test;
Database changed

mysql> create table DummyTest (
    ->    col1 char(10),
    ->    col2 decimal(5,2),
    ->    col3 varchar(32)
    ->    ) engine=DUMMY
    -> ;
Query OK, 0 rows affected (0,04 sec)

mysql> show tables;
+----------------+
| Tables_in_Test |
+----------------+
| DummyTest      |
+----------------+
1 row in set (0,00 sec)

mysql> describe DummyTest;
+-------+--------------+------+-----+---------+-------+
| Field | Type         | Null | Key | Default | Extra |
+-------+--------------+------+-----+---------+-------+
| col1  | char(10)     | YES  |     | NULL    |       |
| col2  | decimal(5,2) | YES  |     | NULL    |       |
| col3  | varchar(32)  | YES  |     | NULL    |       |
+-------+--------------+------+-----+---------+-------+
3 rows in set (0,00 sec)

mysql>

Let's look at the folder and see what happened on the file-system-level:
august@AMD4:~/MySQL/data/Test$ ls -l
insgesamt 16
-rw-rw---- 1 august august   65 Nov 20 16:50 db.opt
-rw-rw---- 1 august august 8620 Nov 20 16:53 DummyTest.frm
august@AMD4:~/MySQL/data/Test$ 
As you can see there are only 2 files: one contains internal-information about the database Test and the other contains create-information about the table DummyTest. There is no data- or index-file, it's really a silly engine.

Back to MySQL we can play with this table:
mysql> insert into DummyTest values ( 'ABC', 5.2, 'Test' );
Query OK, 1 row affected (0,00 sec)

mysql> select * from DummyTest;
Empty set (0,00 sec)

mysql> select count(*) from DummyTest;
+----------+
| count(*) |
+----------+
|        0 |
+----------+
1 row in set (0,00 sec)

mysql> delete from DummyTest where col1='ABC';
Query OK, 0 rows affected (0,00 sec)

mysql> update DummyTest set col2=7.2 where col3 = 'Test';
Query OK, 0 rows affected (0,01 sec)
Rows matched: 0  Changed: 0  Warnings: 0

mysql>

And because we do not need this engine we do some cleanup:
mysql> truncate table DummyTest;
Query OK, 0 rows affected (0,00 sec)

mysql> drop table DummyTest;
Query OK, 0 rows affected (0,00 sec)

mysql> show tables;
Empty set (0,00 sec)

mysql> drop database Test;
Query OK, 0 rows affected (0,06 sec)

mysql> uninstall plugin DUMMY;
Query OK, 0 rows affected (0,03 sec)

mysql> quit
Bye
august@AMD4:~/MySQL/pgm$ 

That's all. As you can see this silly engine was integrated into the MySQL-server. The steps needed to integrate it into the MariaDB-server are similar but I didn't test this.

the code explained

Thanks to OOP and inheritance our little engine includes all the code needed for a storage engine.

To do it manually: in Eclipse I went to the folder storage and created a new folder: dummy
In this folder I created a new file ha_dummy.h as a Header File and also a new file ha_dummy.cc as Source File.

Next I entered all the code you will see in the file ha_dumm.h and ha_dumm.cc

a bit more detailed

The class ha_dummy is derived from the class handler as you can see in the header-file.
We must implement the functions included in the definition of our class because they are defined as pure virtual functions in handler.h. In this engine they are implemented as functions with simply a return-value of OK and nothing more, but there is one exception: in handler.h the function write_row() is defined as returning the constant "unknown command". One does not need to implement this function but in this case an INSERT-statement will result in an error-message. For behaving like the NULL-device it should accept everything (and throw it away) so I implemented this function.

One needs at least 3 more functions which are outside the scope of this class. These are needed for initialising the plugin, creating the class and destroying the plugin. You will see a structure at the end of the file ha_dummy.cc, where two of them are given to the server-code. The 3rd function is given back in the init-code.

Next I switched to another storage engine in the storage-folder and copied the file CMakeList.txt to the dummy-folder. Open this file and change all references of the (old) storage engine to dummy. I took the file from the example-engine, copied it to the dummy-folder and modified this file so that it now looks like this:
SET(DUMMY_PLUGIN_DYNAMIC "ha_dummy")
SET(DUMMY_SOURCES ha_dummy.cc ha_dummy.h)
MYSQL_ADD_PLUGIN(dummy ${DUMMY_SOURCES} STORAGE_ENGINE MODULE_ONLY)
this means changing all occurrences of EXAMPLE into DUMMY (and example into dummy). Save this file.
another hint: do not modify the file in the example-folder! Take the newly copied file from the dummy-folder.

For generating the MAKE-files I switched to a console and entered the line as described above.

Then I switched back to Eclipse, recompiled the source and this storage engine is included in the server. And then I did the checks as described above.