PYTHON > Python et MySQL

pip install pymysql

Database Connection

Before connecting to a MySQL database, make sure of the following points −

• You have created a database TESTDB.
• You have created a table EMPLOYEE in TESTDB.
• This table has fields FIRST_NAME, LAST_NAME, AGE, SEX and INCOME.
• User ID "testuser" and password "test123″ are set to access TESTDB.
• Python module PyMySQL is installed properly on your machine.
• You have gone through MySQL tutorial to understand MySQL Basics.

Following is an example of connecting with MySQL database "TESTDB" −

#!/usr/bin/python3
import pymysql

# Open database connection
db = pymysql.connect("localhost","testuser","test123","TESTDB" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# execute SQL query using execute() method.
cursor.execute("SELECT VERSION()")

# Fetch a single row using fetchone() method.
data = cursor.fetchone()
print ("Database version : %s " % data)

# disconnect from server
db.close()

While running this script, it produces the following result.

Database version : 5.5.20-log

If a connection is established with the datasource, then a Connection Object is returned and saved into db for further use, otherwise db is set to None. Next, db object is used to create a cursor object, which in turn is used to execute SQL queries. Finally, before coming out, it ensures that the database connection is closed and resources are released.

Creating Database Table

Once a database connection is established, we are ready to create tables or records into the database tables using execute method of the created cursor.

Example

Let us create a Database table EMPLOYEE −

#!/usr/bin/python3
import pymysql

# Open database connection
db = pymysql.connect("localhost","testuser","test123","TESTDB" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# Drop table if it already exist using execute() method.
cursor.execute("DROP TABLE IF EXISTS EMPLOYEE")

# Create table as per requirement
sql = """CREATE TABLE EMPLOYEE (
   FIRST_NAME  CHAR(20) NOT NULL,
   LAST_NAME  CHAR(20),
   AGE INT,
   SEX CHAR(1),
   INCOME FLOAT )"""

cursor.execute(sql)

# disconnect from server
db.close()

INSERT Operation

The INSERT Operation is required when you want to create your records into a database table.

Example

The following example, executes SQL INSERT statement to create a record in the EMPLOYEE table −

#!/usr/bin/python3
import pymysql

# Open database connection
db = pymysql.connect("localhost","testuser","test123","TESTDB" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# Prepare SQL query to INSERT a record into the database.
sql = """INSERT INTO EMPLOYEE(FIRST_NAME,
   LAST_NAME, AGE, SEX, INCOME)
   VALUES ('Mac', 'Mohan', 20, 'M', 2000)"""
try:
   # Execute the SQL command
   cursor.execute(sql)
   # Commit your changes in the database
   db.commit()
except:
   # Rollback in case there is any error
   db.rollback()

# disconnect from server
db.close()

The above example can be written as follows to create SQL queries dynamically −

#!/usr/bin/python3
import pymysql

# Open database connection
db = pymysql.connect("localhost","testuser","test123","TESTDB" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# Prepare SQL query to INSERT a record into the database.
sql = "INSERT INTO EMPLOYEE(FIRST_NAME, \
   LAST_NAME, AGE, SEX, INCOME) \
   VALUES ('%s', '%s', '%d', '%c', '%d' )" % \
   ('Mac', 'Mohan', 20, 'M', 2000)
try:
   # Execute the SQL command
   cursor.execute(sql)
   # Commit your changes in the database
   db.commit()
except:
   # Rollback in case there is any error
   db.rollback()

# disconnect from server
db.close()

Example

The following code segment is another form of execution where you can pass parameters directly −

..................................
user_id = "test123"
password = "password"

con.execute('insert into Login values("%s", "%s")' % \
             (user_id, password))
..................................

READ Operation

READ Operation on any database means to fetch some useful information from the database.

Once the database connection is established, you are ready to make a query into this database. You can use either fetchone() method to fetch a single record or fetchall() method to fetch multiple values from a database table.

• fetchone() − It fetches the next row of a query result set. A result set is an object that is returned when a cursor object is used to query a table.
• fetchall() − It fetches all the rows in a result set. If some rows have already been extracted from the result set, then it retrieves the remaining rows from the result set.
• rowcount − This is a read-only attribute and returns the number of rows that were affected by an execute() method.

Example

The following procedure queries all the records from EMPLOYEE table having salary more than 1000 −

#!/usr/bin/python3
import pymysql

# Open database connection
db = pymysql.connect("localhost","testuser","test123","TESTDB" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# Prepare SQL query to INSERT a record into the database.
sql = "SELECT * FROM EMPLOYEE \
      WHERE INCOME > '%d'" % (1000)
try:
   # Execute the SQL command
   cursor.execute(sql)
   # Fetch all the rows in a list of lists.
   results = cursor.fetchall()
   for row in results:
      fname = row[0]
      lname = row[1]
      age = row[2]
      sex = row[3]
      income = row[4]
      # Now print fetched result
      print ("fname = %s,lname = %s,age = %d,sex = %s,income = %d" % \
         (fname, lname, age, sex, income ))
except:
   print ("Error: unable to fetch data")

# disconnect from server
db.close()

Output

This will produce the following result −

fname = Mac, lname = Mohan, age = 20, sex = M, income = 2000

Update Operation

UPDATE Operation on any database means to update one or more records, which are already available in the database.

The following procedure updates all the records having SEX as ‘M’. Here, we increase the AGE of all the males by one year.

Example

#!/usr/bin/python3
import pymysql

# Open database connection
db = pymysql.connect("localhost","testuser","test123","TESTDB" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# Prepare SQL query to UPDATE required records
sql = "UPDATE EMPLOYEE SET AGE = AGE + 1
                          WHERE SEX = '%c'" % ('M')
try:
   # Execute the SQL command
   cursor.execute(sql)
   # Commit your changes in the database
   db.commit()
except:
   # Rollback in case there is any error
   db.rollback()

# disconnect from server
db.close()

DELETE Operation

DELETE operation is required when you want to delete some records from your database. Following is the procedure to delete all the records from EMPLOYEE where AGE is more than 20 −

Example

#!/usr/bin/python3
import pymysql

# Open database connection
db = pymysql.connect("localhost","testuser","test123","TESTDB" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# Prepare SQL query to DELETE required records
sql = "DELETE FROM EMPLOYEE WHERE AGE > '%d'" % (20)
try:
   # Execute the SQL command
   cursor.execute(sql)
   # Commit your changes in the database
   db.commit()
except:
   # Rollback in case there is any error
   db.rollback()

# disconnect from server
db.close()

Performing Transactions

Transactions are a mechanism that ensures data consistency. Transactions have the following four properties −

• Atomicity − Either a transaction completes or nothing happens at all.
• Consistency − A transaction must start in a consistent state and leave the system in a consistent state.
• Isolation − Intermediate results of a transaction are not visible outside the current transaction.
• Durability − Once a transaction was committed, the effects are persistent, even after a system failure.

The Python DB API 2.0 provides two methods to either commit or rollback a transaction.

Example

You already know how to implement transactions. Here is a similar example −

# Prepare SQL query to DELETE required records
sql = "DELETE FROM EMPLOYEE WHERE AGE > '%d'" % (20)
try:
   # Execute the SQL command
   cursor.execute(sql)
   # Commit your changes in the database
   db.commit()
except:
   # Rollback in case there is any error
   db.rollback()

COMMIT Operation

Commit is an operation, which gives a green signal to the database to finalize the changes, and after this operation, no change can be reverted back.

Here is a simple example to call the commit method.

db.commit()

ROLLBACK Operation

If you are not satisfied with one or more of the changes and you want to revert back those changes completely, then use the rollback() method.

Here is a simple example to call the rollback() method.

db.rollback()

Disconnecting Database

To disconnect the Database connection, use the close() method.

db.close()

If the connection to a database is closed by the user with the close() method, any outstanding transactions are rolled back by the DB. However, instead of depending on any of the DB lower level implementation details, your application would be better off calling commit or rollback explicitly.

Handling Errors

There are many sources of errors. A few examples are a syntax error in an executed SQL statement, a connection failure, or calling the fetch method for an already canceled or finished statement handle.

The DB API defines a number of errors that must exist in each database module. The following table lists these exceptions.

Sr.No.	Exception & Description
1	Warning  Used for non-fatal issues. Must subclass StandardError.
2	Error  Base class for errors. Must subclass StandardError.
3	InterfaceError  Used for errors in the database module, not the database itself. Must subclass Error.
4	DatabaseError  Used for errors in the database. Must subclass Error.
5	DataError  Subclass of DatabaseError that refers to errors in the data.
6	OperationalError  Subclass of DatabaseError that refers to errors such as the loss of a connection to the database. These errors are generally outside of the control of the Python scripter.
7	IntegrityError  Subclass of DatabaseError for situations that would damage the relational integrity, such as uniqueness constraints or foreign keys.
8	InternalError  Subclass of DatabaseError that refers to errors internal to the database module, such as a cursor no longer being active.
9	ProgrammingError  Subclass of DatabaseError that refers to errors such as a bad table name and other things that can safely be blamed on you.
10	NotSupportedError  Subclass of DatabaseError that refers to trying to call unsupported functionality.

Your Python scripts should handle these errors, but before using any of the above exceptions, make sure your MySQLdb has support for that exception. You can get more information about them by reading the DB API 2.0 specification.

 

 

—

he Entree

With the boring stuff out of the way, let’s dig into some goodness. We’ll start off with a basic use case: selecting all rows from a table:

...
class Database:
    """Database connection class."""

    ...

    def run_query(self, query):
        """Execute SQL query."""
        try:
            self.open_connection()
            with self.conn.cursor() as cur:
                records = []
                cur.execute(query)
                result = cur.fetchall()
                for row in result:
                    records.append(row)
                cur.close()
                return records
        except pymysql.MySQLError as e:
            print(e)
        finally:
            if self.conn:
                self.conn.close()
                self.conn = None
                print('Database connection closed.')

run_query() tries to open a connection using the open_connection() function we created earlier. With that connection open, we’re free to run whichever queries we want.

Our function is passed a parameter called query, which represents whichever SQL query we want to run (AKA: SELECT * FROM [table]). To execute such a query, we need to open a cursor, which we do with this line:

with self.conn.cursor() as cur:
    ...

We’re able to run all the queries we want while cur is open. Let’s kick it up a notch.

Selecting Rows of Data

Now with cur open, we can call a few methods against cur.

• cur.execute([QUERY]): Calling execute() on cur will run a query inside our cursor object.
• cur.fetchall(): After running a query which results in rows, we can see all the rows returned by our query by calling fetchall() on cur. As you can see, we should only execute one query per cursor. Otherwise, we’d be writing over the results of our previous query over and over. Attempting to print the result of .fetchall() will result in a single integer, which represents the number of rows fetched.
• cur.fetchone(): Unlike fetchall(), fetchone() only fetches the first row returned by our query. If there are instances where we know only one record should be returned, .fetchone() should be used.
• cur.close(): When we’re finally done with our query, we should close the cursor with close().

run_query() assumes that the query we’re passing is a SELECT query, which is why we’re saving the results to an array. If we’re running an UPDATE or DELETE query, this wouldn’t make much sense since nothing would be returned.

Updating Rows of Data

What if we’re not selecting data, but rather modifying it? Check our how our function changes:

def run_query(self, query):
   """Execute SQL query."""
   try:
      self.open_connection()
      with self.conn.cursor() as cur:
   result = cur.execute(query)
   self.conn.commit()
   affected = f"{cur.rowcount} rows affected."
   cur.close()
   return affected
   ...

There are a few new things here:

• conn.commit(): Running commit() actually commits the changes in our query. If you forget this, changes to your data won’t actually be saved (this goes for DELETE and INSERT statements as well).
• cur.rowcount: Returns the number of rows affected by a query which changes data.

Combining the Above

So now we have a version of run_query() which expects SELECT statements, and another which expects mutations. How could we make this function smart enough to handle either case and return the result which makes the most sense? How about checking to see if the word SELECT exists in the query before executing?:

...
class Database:
    """Database connection class."""

    ...

    def run_query(self, query):
        """Execute SQL query."""
        try:
            self.open_connection()
            with self.conn.cursor() as cur:
                if 'SELECT' in query:
                    records = []
                    cur.execute(query)
                    result = cur.fetchall()
                    for row in result:
                        records.append(row)
                    cur.close()
                    return records
                else:
                    result = cur.execute(query)
                    self.conn.commit()
                    affected = f"{cur.rowcount} rows affected."
                    cur.close()
                    return affected
        except pymysql.MySQLError as e:
            print(e)
        finally:
            if self.conn:
                self.conn.close()
                self.conn = None
                logging.info('Database connection closed.')

Now we have a function that handles both scenarios! Of course, this implementation isn’t exactly foolproof: if one of your columns happens to named SELECT (or something), you might run into some problems. Don’t do that, I suppose.

For Dessert

Hopefully you’ve found our little dinner date to be useful. If you’re looking for some copy & paste source to get started with PyMySQL on our own, feel free to check out the source code for this up on Github here. If that’s too much, feel free to copy + paste the below:

import sys
import pymysql
import logging

class Database:
    """Database connection class."""

    def __init__(self, config):
        self.host = config.db_host
        self.username = config.db_user
        self.password = config.db_password
        self.port = config.db_port
        self.dbname = config.db_name
        self.conn = None

    def open_connection(self):
        """Connect to MySQL Database."""
        try:
            if self.conn is None:
                self.conn = pymysql.connect(self.host,
                                            user=self.username,
                                            passwd=self.password,
                                            db=self.dbname,
                                            connect_timeout=5)
        except pymysql.MySQLError as e:
            logging.error(e)
            sys.exit()
        finally:
            logging.info('Connection opened successfully.')

    def run_query(self, query):
        """Execute SQL query."""
        try:
            self.open_connection()
            with self.conn.cursor() as cur:
                if 'SELECT' in query:
                    records = []
                    cur.execute(query)
                    result = cur.fetchall()
                    for row in result:
                        records.append(row)
                    cur.close()
                    return records
                else:
                    result = cur.execute(query)
                    self.conn.commit()
                    affected = f"{cur.rowcount} rows affected."
                    cur.close()
                    return affected
        except pymysql.MySQLError as e:
            print(e)
        finally:
            if self.conn:
                self.conn.close()
                self.conn = None
                logging.info('Database connection closed.')

We thank you all for joining us in this adventure of tantalizing treats. May your data be clean and your stomachs full.

Bon appétit.

 

—

Examples

CRUD

The following examples make use of a simple table

CREATE TABLE `users` (
    `id` int(11) NOT NULL AUTO_INCREMENT,
    `email` varchar(255) COLLATE utf8_bin NOT NULL,
    `password` varchar(255) COLLATE utf8_bin NOT NULL,
    PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin
AUTO_INCREMENT=1 ;

import pymysql.cursors

# Connect to the database
connection = pymysql.connect(host='localhost',
                             user='user',
                             password='passwd',
                             db='db',
                             charset='utf8mb4',
                             cursorclass=pymysql.cursors.DictCursor)

try:
    with connection.cursor() as cursor:
        # Create a new record
        sql = "INSERT INTO `users` (`email`, `password`) VALUES (%s, %s)"
        cursor.execute(sql, ('webmaster@python.org', 'very-secret'))

    # connection is not autocommit by default. So you must commit to save
    # your changes.
    connection.commit()

    with connection.cursor() as cursor:
        # Read a single record
        sql = "SELECT `id`, `password` FROM `users` WHERE `email`=%s"
        cursor.execute(sql, ('webmaster@python.org',))
        result = cursor.fetchone()
        print(result)
finally:
    connection.close()

This example will print:

{'password': 'very-secret', 'id': 1}

 

—

 

 

—

 

 

—