Text_factory that can support 8-bit bytestrings

I wrote a Python script that will download a webpage, extract a portion of the text displayed on the page and write the extracted portion to an SQLite database. When I ran the script, I saw the message below displayed:

You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.

I had created the following function to establish the connection to the SQLITE 3 database:

def create_connection (db_file):
   """ Create a database connection to an SQL database
       Return connection object or none """
   try:
      conn = sqlite3.connect(db_file)
      return conn
   except Exception as e:
      print(e)
   return None

I called the function using the code shown below:

   conn = create_connection(dbname)
   if conn is not None:
     # create posts table
     create_table(conn,sql_create_posts_table)
   else:
      print("Error! cannot create the database connection.")

The only change I needed to make to resolve the problem was to add one additional line to the function that creates the database connection. I had to add conn.text_factory = str.

Mastering Python
Mastering Python
1x1 px

def create_connection (db_file):
   """ Create a database connection to an SQL database
       Return connection object or none """
   try:
      conn = sqlite3.connect(db_file)
      conn.text_factory = str
      return conn
   except Exception as e:
      print(e)
   return None

Once I added that line, I no longer saw the text_factory message displayed and when I checked the SQLite database aftwards using the sqlite3 utility to examine the table into which I was attempting to insert a row containing text extracted from the downloaded webpage, I found the appropriate entry in the table.

Related articles:

  1. Downloading a web page with Python
  2. Importing data from a text file into an SQLite database with Python