MoonPoint Support Logo


Shop Amazon Warehouse Deals - Deep Discounts on Open-box and Used ProductsAmazon Warehouse Deals

Advanced Search
Sun Mon Tue Wed Thu Fri Sat
17 18 19 20 21
22 23 24 25 26 27 28
29 30 31        
OctNov Dec

Sat, Oct 14, 2017 8:59 pm

Text_factory that can support 8-bit bytestrings

I wrote a Python script that will download a webpage, extract a portion of the text displayed on the page and write the extracted portion to an SQLite database. When I ran the script, I saw the message below displayed:

You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.

I had created the following function to establish the connection to the SQLITE 3 database:

def create_connection (db_file):
   """ Create a database connection to an SQL database
       Return connection object or none """
      conn = sqlite3.connect(db_file)
      return conn
   except Exception as e:
   return None

[ More Info ]

[/languages/python] permanent link

Sat, Oct 07, 2017 10:59 pm

Determining the day of year value using Python

I sometimes need to determine the day of the year corresponding to today's date. The day of year starts with January 1 as day 1 and for 2017, December 31 is day 365. You can find sites online that will provide those values, e.g. Day Numbers for 2017 or NOAA's DOY Calendar. Or, on a system with Python installed, e.g., Linux or OS X, you can use the datetime module to obtain the day of the year corresponding to the current date as shown below:

$ python
Python 2.7.10 (default, Oct 23 2015, 19:19:21) 
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime
>>> exit()

Since today is October 7 of the year 2017, the day of the year is 280.

If you want to obtain the day of year (DOY) for another date, you can use, month, day).timetuple().tm_yday where year is the relevant year, month is the month, and day is the day of interest. E.g., March 1, 2017 is DOY 60:

>>> import datetime
>>>, 3, 1).timetuple().tm_yday

[/languages/python] permanent link

Sun, Oct 01, 2017 10:53 pm

Finding Gmail SMTP entries in Sendmail log files

I wanted to determine how many connections I was receiving per day from Gmail Simple Mail Transfer Protocol (SMTP) servers to my email server running Sendmail on a CentOS Linux system and the IP addresses of the Gmail servers that were sending email to users on my server. So I created a simple Python script to search for lines in the maillog file, /var/log/maillog for any lines containing "relay" and "" on the same line, since the Gmail servers are in Google's domain.

[ More Info ]

[/languages/python] permanent link

Tue, Sep 12, 2017 10:58 pm

Adding an item to a list in Python without duplicates

If you have a list in a Python script and wish to add new items to the list, you can use listname.append(newitem) where listname is the name of the list and newitem is the item you wish to add to the list. If you wish to add new items to the list only if they don't already exist in the list, you can avoid adding an existing entry to the list by checking if the entry is already in the list as shown below:

filenameList = []
if filename not in filenameList:

If you wish to know the length of a list, i.e., how many items are on the list, you can use print len(listname) where listname is the name of your list.

[/languages/python] permanent link

Fri, Sep 08, 2017 10:21 pm

Display a webpage from a Python script

If you just want to display text on a webpage from a Python script, a simple way to print to a web page is shown in the script below:


print "Content-type: text/plain\n"
print "Hello world"

If you want to display HTML code, you can use Python code like the following:


print "Content-type: text/html"
print """<!DOCTYPE HTML>
<html lang="en">
<h1>Hello World!</h1>

[ More Info ]

[/languages/python] permanent link

Sat, Aug 26, 2017 9:29 pm

Querying a MariaDB or MySQL database using Python

On a CentOS Linux system, I have a MariaDB (MariaDB is a fork of MySQL) database named "Books" that contains a table named "Amazon" where I've stored information on books that includes their Amazon Standard Identification Number (ASIN). I want to be able to run a Python script to which I provide a word as a command line argument and have the script find all books in that database table that have a title containing that word and to print the title, which is stored in a column named "Description" along with the ASIN, which is stored in a column named "ASIN". To do so, I use a script with the following code:


import mysql.connector, sys

def executeSQL(conn, queryTerm):
    cursor = conn.cursor()
    query = "select ASIN, Description from Amazon where Description like " + queryTerm
    result = cursor.fetchall()    
    for row in result:
       print row[0], row[1]

hostname = 'localhost'
username = 'myAcctName'
password = 'ThePassword'
database = 'Books'

except IndexError:
   print "Error - missing query word! Usage ./findBook queryWord"
   queryWord = "'%" + sys.argv[1] + "%'" 

connection = mysql.connector.connect( host=hostname, user=username, passwd=password, db=database )

[ More Info ]

[/languages/python] permanent link

Wed, Aug 23, 2017 11:36 pm

Determining a file's type from within a Python script

I needed a way to determine a file's type within a Python script when I can't rely on the file's extension to determine the file format. I'll be running the script on a MacBook Pro laptop running the OS X El Capitan operating system. OS X/macOS, like Linux, comes with the file command, so I could run that command at a shell prompt to have the utility check the magic number in the files I'm interested in, but I want to do some additional processing of the files within the Python script, so I want to perform the format check within Python. Python provides the subprocess module that provides the capability to "spawn new processes, connect to their input/output/error pipes, and obtain their return codes." So I can call the file utility from within Python using that module. To get the results from running a shell command, you use suprocess.Popen(). You can then set a variable to hold the results of .communicate() and print the contents of that variable as shown below. The script expects the name of the file to be checked to be provided as an argument on the command line.


import subprocess as sub, sys

except IndexError:
   print "Error - missing input file name! Usage ./ infile"
   fileName = sys.argv[1]

p = sub.Popen(['file',fileName],stdout=sub.PIPE,stderr=sub.PIPE)
output, errors = p.communicate()
print output

[ More Info ]

[/languages/python] permanent link

Mon, Aug 21, 2017 11:15 pm

Find all occurrences of a string in a file using Python

I need to download and view Excel workbooks, Portable Document Format (PDF), and sometimes other types of documents related to work requests from a website. When I view a webpage for a particular work request, some of the documents may have been posted weeks or months ago while some are more recent, but when I download them they all get the timestamp of the time I downloaded them, but I would like to change the timestamps to match the timestamps on the website. On the webpage for a particular work request, I see the following information for files:

Type Name Site Modified Modified By
xlsm icon CRQ000000884164_PDSO -None- 3/6/2017 4:53 PM Smith, Gary
pdf icon CRQ000000884164_PDSO -None- 3/6/2017 4:52 PM Smith, Gary
pdf icon CRQ000000884164_DCS -None- 6/12/2017 9:29 AM Doe, Mike
xlsm icon CRQ 884164_SDP -None- 6/12/2017 9:30 AM Doe, Mike

[ More Info ]

[/languages/python] permanent link

Fri, Aug 18, 2017 10:41 pm

Importing data from a text file into an SQLite database with Python

I need to track work requests that are associated with particular projects. The work requests are submitted through a website, but I want to be able to run queries on the data that aren't available to me from the website. So I decided to add a projects table to an existing SQLite database I have on my MacBook Pro laptop. I didn't want to manually enter a list of about 200 projects into the table, however, so I copied the projects list from the website into a text file. The webpage that lists all of the projects includes projects that are no longer valid and I want to include those in my database table, but note that they are longer valid. When I copy the data from the web page, I have lines like the following ones:

450 SN/GN Tech  Edit Delete
ACE     Edit Delete
ADO     Edit Undelete
AGO     Edit Delete
AGS     Edit Delete
AIM     Edit Delete
Artemis         Edit Undelete
ASF     Edit Delete
ATSC    Edit Undelete
AXAF    Edit Undelete
BATSE/Gro       Edit Undelete
CANDOS  Edit Undelete
CARA    Edit Delete
CD Manager      Edit Delete
CMF     Edit Undelete
CMOC    Edit Delete
CMS     Edit Undelete

The entries that are no longer valid have "Undelete" at the end of the line indicating the projects were deleted, but they can be undeleted.

I created a table in the database with the following columns all of which contain text data:

NameDescription AddedRemoved DeletedNotes

[ More Info ]

[/languages/python] permanent link

Thu, Jul 20, 2017 10:49 pm

Passing a parameter to a Python script from a web page

I have a Python script that I use to process a copy of a webpage downloaded from a website and stored on my MacBook Pro laptop's hard drive to produce a CSV file from the data within that file. I was running the script from a command-line interface (CLI), i.e., a Terminal window, on the system by issuing a command like ./ inputfile outputfile where inputfile and outputfile were the file names and locations of the file holding the data and the output CSV file, respectively. I wanted to execute that script from a link on a web page, instead, so I needed a way to pass the arguments I had been passing on the command line to the Python script in the URL that I'd specify as the link on the web page. One way that you can do that for Python is explained at [Tutor] Passing Arguments to a Remote CGI Script where the following sample Python script is shown:

"""test.cgi  --- a small CGI program to show how one can pass parameters.

import cgi
fs = cgi.FieldStorage()

print "Content-type: text/plain\n"
for key in fs.keys():
    print "%s = %s" % (key, fs[key].value)

[ More Info ]

[/languages/python] permanent link

Once You Know, You Newegg AliExpress by

Shop Amazon Local - Subscribe to Deals in Your Neighborhood

Valid HTML 4.01 Transitional

Privacy Policy   Contact

Blosxom logo