I read everything I could find on MySQL Performance Tuning and innodb. Here’s a summary of what helped me:

1. The book High Performance MySQL is good, but only gets you so far.

2. The blog MySQL Performance Blog (this link is to their posts tagged ‘innodb’) was the most useful overall resource I found on the net. They go into detail on a lot of innodb tuning issues. It gets ‘ranty’ at times, but overall it’s great. Here’s another link there on InnoDB Performance Optimization Basics that’s good.

3. The last main thing I did to learn it was to simply read the MySQL Docs themselves. I read how every last parameter works, changed them on my server and then did some basic profiling. After a while you figure out what works by running big queries and seeing what happens. Here’s a good place to start:
InnoDB Performance Tuning and Troubleshooting

In the end, it’s just experimenting and working through things until you gain enough knowledge to know what works.

Please share any other ideas, pointers or resources you know of in the comments — Thanks!

Moreover, each can be signed or unsigned. If your value can’t ever be negative (for example, for an index), then you should probably make your values unsigned.

DROP TABLE `foo`.`example_03`;
CREATE TABLE IF NOT EXISTS `foo`.`example_03` (
 f_01  TINYINT,           -- from -128 to 127
 f_02  TINYINT UNSIGNED,  -- from 0 to 256
 f_03  SMALLINT,          -- from -32,768 to 32,767
 f_04  SMALLINT UNSIGNED, -- from 0 to 65,535
 f_05  MEDIUMINT,         -- from -8,388,608 to 8,388,607
 f_06  MEDIUMINT UNSIGNED, -- from 0 to 16,777,215
 f_07  INT,               -- from -2,147,483,648 to 2,147,483,647
 f_08  INT UNSIGNED,      -- from 0 to 4,294,967,295
 f_09  BIGINT,
              -- from -9,223,372,036,854,775,808
              -- to  9,223,372,036,854,775,807
 f_10  BIGINT UNSIGNED    -- from 0 to 18,446,744,073,709,551,615
);

Click here to go to the table of contents for all MySQL Programming Example Code

(Note that we’re also using “CREATE TABLE IF NOT EXISTS” so an error doesn’t get thrown if the table already exists when we try to create it.)

We’ve added comments as well, though we’ll explain those after.

/*
 Create the table foo.example_02 if it doesn't exist.
*/
CREATE TABLE IF NOT EXISTS `foo`.`example_02` (
 `field_1` VARCHAR(255) NOT NULL DEFAULT '',
 `field_2` CHAR(20) NOT NULL default 'default_text',
 `field_3` INT(10) DEFAULT NULL,
 `field_4` BIGINT(20),
 `field_5` DATETIME          -- embed a comment in the definition like this...
);

Note that if you don’t specify NULL or NOT NULL, columns by default can contain NULL values. And not specifying a default is the same as specifying ‘DEFAULT NULL’ since MySQL uses NULL as the default value in both cases.

Note also that I’ve added comments to the declaration. Comments come in two types — multi-line comments, and single-line comments.

/*
      This is a multi-line comment
*/

-- This is single-line comment 

select foo from bar;   -- This means the rest of this line is a comment

Click here to go to the table of contents for all MySQL Programming Example Code

CREATE TABLE `foo`.`example_01` (
`field_1`         VARCHAR(255),
`field_2`         CHAR(10),
`field_3`         INT(10),
`field_4`         BIGINT(20),
`field_5`         DATETIME
);

Notice that the ‘back tic’s on each column definition (the little ` characters) are optional in most cases. I use them for reasons I’ll discuss later on.

Also, for most columns you see a number in parentheses after the name of the type — that is the default size for displaying that value in a select statement.

The actual size of the value you store in that column isn’t impacted by changing that number. Changing INT(10) to INT(12) won’t change the size of the data you can store in that column.

Click here to go to the table of contents for all MySQL Programming Example Code

Sometimes when using Active Record you may want to create a database connection to a database other than the default database specified in your database.yml file.

When this happens, the easiest way to do it is to use the Class-level convenience method on ActiveRecord::Base. Like this:

Imagine your database.yml entry looks like this:

development_foo:
  adapter: mysql
  encoding: utf8
  database: foo
  pool: 5
  username: my_user
  password: my_pass
  socket: /tmp/mysql.sock

Now you want to create active record models that use that database connection. Here’s how:

# 'Bar' model points to the table 'bars' in the database 'foo'

class Bar < ActiveRecord::Base

  # specify the database.yml entry
  $db_config = 'development_foo'

  # Fetch the database.yml configuration
  $config = ActiveRecord::Base.configurations[$db_config]

 # Now we can establish a connection to that database and
 # this ActiveRecord model class will point to that database.
  establish_connection $config

end

It’s simple!

Copyright Kevin Bedell, 2009
All this code can be freely used under the terms of the MIT License.

What is a cursor and when should you use one?

A ‘cursor’ in MySQL is essentially just the result set that’s returned from a query. Using a cursor allows you to iterate, or step through, the results of a query and perform certain operations on each row that’s returned.

Think of them like holders for any data you  might process in a loop. As you’ll see below, the example here runs a query then loops through the results.

Cursors have some drawbacks and you don’t want to use them everywhere. An appropriate use for a cursor might be when you need to step through the results of a query and then perform operations on multiple tables for each row.

Another candidate for a cursor is where some steps of the processing are optional and only need to be performed for certain rows. The cursor allows you to iterate through the result set and then perform the additional processing only on the rows that require it.

When should cursors be avoided?

Cursors should be avoided generally whenever you can write a SQL statement that processes all rows in a query at once — and you’ll be surprised at how often this is. If you try, many procedures with cursors in them can be refactored to eliminate the cursor by using a bit more complex SQL.

This is an advantage usually for a couple reasons. First, by processing all rows at once instead of looping through them one at a time, it’s generally faster. Second, cursors can make some issues more complex to debug

MySQL Cursor Sample Code

Here’s an example of how to create a MySQL stored procedure that uses a cursor.

I ‘over-commented’ the procedure a bit (if there is such a thing!), but I wanted the code to be relatively self-explanatory for people who just cut/paste it.

The SQL required to setup the tables and data follows the example. Everything runs and is tested. It requires MySQL 5.0 or above since before that there was no support for stored procedures.

First, here’s the sample code. I’ll follow after with some explanation.

/*
 *      Procedure Name  :  usp_cursor_example
 *      Database/Schema :  foo
 *
 *      Description:
 *          An example of a MySQL stored procedure that uses a cursor
 *
 *
 *      Tables Impacted :
 *         foo.friend_status - read-only
 *         DDL? - None
 *         DML? - Only Selects
 *
 *      Params:
 *         name_in - the name of the friend to search for
 *
 *      Revision History:
 *
 *         Date:          Id:        Comment:
 *         2009/03/01     kbedell    Original
 *
 *    Copyright (c) 2009 Kevin Bedell
 *    Can be resused under terms of the 'MIT license'.
 *
 *    To test:
 *      - Multiple records: call foo.usp_cursor_example('John');
 *      - One record:       call foo.usp_cursor_example('Julie');
 *      - Zero records:     call foo.usp_cursor_example('Waldo');
 *
 */
DROP PROCEDURE IF EXISTS  `foo`.`usp_cursor_example`;
DELIMITER $$
CREATE DEFINER=`root`@`localhost` PROCEDURE `foo`.`usp_cursor_example`(
  IN name_in VARCHAR(255)
)
READS SQL DATA
BEGIN

  /*
    All 'DECLARE' statements must come first
  */

  -- Declare '_val' variables to read in each record from the cursor
  DECLARE name_val VARCHAR(255);
  DECLARE status_update_val VARCHAR(255);

  -- Declare variables used just for cursor and loop control
  DECLARE no_more_rows BOOLEAN;
  DECLARE loop_cntr INT DEFAULT 0;
  DECLARE num_rows INT DEFAULT 0;

  -- Declare the cursor
  DECLARE friends_cur CURSOR FOR
    SELECT
        name
      , status_update
    FROM foo.friend_status
    WHERE name = name_in;

  -- Declare 'handlers' for exceptions
  DECLARE CONTINUE HANDLER FOR NOT FOUND
    SET no_more_rows = TRUE;

  /*
    Now the programming logic
  */

  -- 'open' the cursor and capture the number of rows returned
  -- (the 'select' gets invoked when the cursor is 'opened')
  OPEN friends_cur;
  select FOUND_ROWS() into num_rows;

  the_loop: LOOP

    FETCH  friends_cur
    INTO   name_val
    ,      status_update_val;

    -- break out of the loop if
      -- 1) there were no records, or
      -- 2) we've processed them all
    IF no_more_rows THEN
        CLOSE friends_cur;
        LEAVE the_loop;
    END IF;

    -- the equivalent of a 'print statement' in a stored procedure
    -- it simply displays output for each loop
    select name_val, status_update_val;

    -- count the number of times looped
    SET loop_cntr = loop_cntr + 1;

  END LOOP the_loop;

  -- 'print' the output so we can see they are the same
  select num_rows, loop_cntr;

END
DELIMITER ;

Discussion and Explanation of the MySQL Cursor Example Code

The ‘Declaration’ section

Before any actual programming logic starts, you need to first list all ‘declarations’. This is a requirement of MySQL.

Here’s the declaration section:

  /*
    All 'DECLARE' statements must come first
  */

  -- Declare '_val' variables to read in each record from the cursor
  DECLARE name_val VARCHAR(255);
  DECLARE status_update_val VARCHAR(255);

  -- Declare variables used just for cursor and loop control
  DECLARE no_more_rows BOOLEAN;
  DECLARE loop_cntr INT DEFAULT 0;
  DECLARE num_rows INT DEFAULT 0;

  -- Declare the cursor
  DECLARE friends_cur CURSOR FOR
    SELECT
        name
      , status_update
    FROM foo.friend_status
    WHERE name = name_in;

  -- Declare 'handlers' for exceptions
  DECLARE CONTINUE HANDLER FOR NOT FOUND
    SET no_more_rows = TRUE;

There are 4 sections to the declaration section:

• Declaration of one variable to hold the ‘value’ of each column in the result of the query.

Here, I recommend simply naming each variable by adding ‘_val’ to the end of the actual returned column name. Also, make sure the types match!

• Declaration of variables that are used for counting rows and controlling looping.

Generally, every stored procedure that uses a cursor can always simply have the same set.

In this example we have three variables — a boolean that gets set if we try to read from the cursor when no data is left, and two counters for the number of rows processed (we have two here because the example demonstrates two different methods of counting rows processed — use either of them).

• Declaration of the actual query that will generate the result set to be held by the cursor

The query is not actually invoked until later in the procedure where the cursor is ‘opened’. Also, in this example you can see that the cursor uses a variable in its ‘where clause’ that is passed in as an argument to the procedure.

• Declaration of ‘handlers’ to control execution. These are usually always the same.

This is a standard declaration that sets up processing for when the cursor contains no (or no more) rows.

Opening the cursor and finding the number of rows returned

The next step is to ‘open’ the cursor. This is where the actual query in the cursor is run – and one of the places where cursor programming can get tricky.

  -- 'open' the cursor and capture the number of rows returned
  -- (the 'select' gets invoked when the cursor is 'opened')
  OPEN friends_cur;
  select FOUND_ROWS() into num_rows;

Notice that we select FOUND_ROWS() into our num_rows variable directly after we open the cursor. The FOUND_ROWS() ‘information function’ contains the number of rows found in the last select statement run.

If the query fails or errors for some reason, the ‘open’ call will fail — and it can fail ‘silently’. This is one of the issues with cursors — you need to make sure that your queries work and are solid.

You also need to make sure that they are tolerant to data changing over time. Make sure that if the underlying data in the table changes over time, your query will still work.

Looping through the cursor result set

There are multiple methods for looping in MySQL — I use this method and recommend that you just cut/paste this code for every cursor you write. This method works and you won’t have to think about it.

  the_loop: LOOP

    FETCH  friends_cur
    INTO   name_val
    ,      status_update_val;

    -- break out of the loop if
      -- 1) there were no records, or
      -- 2) we've processed them all
    IF no_more_rows THEN
        CLOSE friends_cur;
        LEAVE the_loop;
    END IF;

    -- the equivalent of a 'print statement' in a stored procedure
    -- it simply displays output for each loop
    select name_val, status_update_val;

    -- count the number of times looped
    SET loop_cntr = loop_cntr + 1;

  END LOOP the_loop;

There are a couple things to point out here.

First, note that the no_more_rows variable gets set automatically when you ‘FETCH’ from the cursor. For this reason you should always test for that condition immediately after you try to read a row.

Note that we have a counter in the loop here as well. This is just another way to count the rows processed. Use what works for you.

Conclusion

In this post, we gave some sample code that can be used/reused to create MySQL Stored Procedures containing a cursor. Best of luck with the code and feel free to post comments below if you find any problems or if you have any suggestions!

Setup code for the sample MySQL Stored Procedure with a Cursor

Here is the setup SQL I used to create the tables/data you need to test the stored procedure and cursor.

/*
 *      Procedure/File Name  :  ex1_setup.sql
 *      Database/Schema      :  foo
 *
 *      Description:
 *          Setup file to support simple cursor example
 *
 *      Tables Impacted :
 *         foo.friend_status - drop/re-create table and populate.
 *
 *      Params:
 *         None.
 *
 *      Revision History:
 *
 *         Date:          Id:        Comment:
 *         2009/03/01     kbedell    Original
 *
 *    Copyright (c) 2009 Kevin Bedell
 *    Can be resused under terms of the 'MIT license'.
 *
 *    To test:
 *      - Run script then: select * from foo.friend_status;
 *
 */
drop table if exists foo.friend_status; 

CREATE TABLE IF NOT EXISTS `foo`.`friend_status` (
    `id`            INTEGER(10) unsigned NOT NULL auto_increment,
    `name`          VARCHAR(255) NOT NULL,
    `status_update` VARCHAR(255) NOT NULL,
    PRIMARY KEY (`id`)
);

insert into foo.friend_status
    (name, status_update)
  values
      ('John',  'Woke up. Guiness for Brkfst.')
    , ('Fred',  'is thinking about joining the circus')
    , ('Erin',  "Getting ready for a job interview")
    , ('Amy',   'at work and dreaming of kittens')
    , ('John',  'Watching Scooby Doo reruns. Guiness for Lunch.')
    , ('Amy',   'dreaming of fuzzy slippers and wedding dresses')
    , ('Julie', 'is hating working two jobs')
    , ('John',  'Out of the shower finally. Guiness for Dinner.')
    , ('Erin',  "if I don't get this job, I'll be asking 'Paper or Plastic?'")
    , ('Amy',   'dreaming of Meeting Mr. Right!')
    , ('Erin',  'Nailed the job interview -- calling John to celebrate!')
    , ('Amy',   'John called -- meeting him at the pub!')
    , ('John',  'Heading out to meet friends for some Guiness!')
;