MoonPoint Support Logo


Shop Amazon Warehouse Deals - Deep Discounts on Open-box and Used ProductsAmazon Warehouse Deals

Advanced Search
Sun Mon Tue Wed Thu Fri Sat
18 19 20 21 22 23
24 25 26 27 28 29 30

Sun, Oct 29, 2017 10:10 pm

Decoding base64-encoded text in a .eml file with Python

I received an email message from someone that contained an attachment that had a .eml filename extension. When I viewed that file, I found the usual email header fields, i.e., "from", "to", "cc", and "subject", but for the body of the message I saw the following:

Content-Type: multipart/alternative;
MIME-Version: 1.0

Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

<text snipped>


I needed to decode the base64 encoded text. Fortunately, Python has a base64 module that can be used for that purpose. So I created the following Python script to decode the encoded portion of the .eml file. The script expects the name of the input file to be provided on the command line and will print an error message and terminate if the file name isn't provided. If the filename is provided, the script will read the file line by line looking for the "Content-Transfer-Encoding: base64" which indicates that after one following blank line the encoded text will commence. The output will be displayed on the screen but can be redirected to a file.


import base64, sys

except IndexError:
   print "Error - missing input file name! Usage", sys.argv[0], "infile"
   fileName = sys.argv[1]

base64_marker = "Content-Transfer-Encoding: base64"
block_of_lines = ""

with open(fileName) as input_data:
    # Skips text before the beginning of the base64 encoded block:
    for line in input_data:
        if line.strip() == 'Content-Transfer-Encoding: base64':
    for line in input_data:  # Skip blank line
    # Reads text until the end of the block:
    for line in input_data:  # Append lines to block_of_lines
        block_of_lines = block_of_lines + line
    print base64.b64decode(block_of_lines)

[/languages/python] permanent link

Sat, Oct 14, 2017 8:59 pm

Text_factory that can support 8-bit bytestrings

I wrote a Python script that will download a webpage, extract a portion of the text displayed on the page and write the extracted portion to an SQLite database. When I ran the script, I saw the message below displayed:

You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.

I had created the following function to establish the connection to the SQLITE 3 database:

def create_connection (db_file):
   """ Create a database connection to an SQL database
       Return connection object or none """
      conn = sqlite3.connect(db_file)
      return conn
   except Exception as e:
   return None

[ More Info ]

[/languages/python] permanent link

Sat, Oct 07, 2017 10:59 pm

Determining the day of year value using Python

I sometimes need to determine the day of the year corresponding to today's date. The day of year starts with January 1 as day 1 and for 2017, December 31 is day 365. You can find sites online that will provide those values, e.g. Day Numbers for 2017 or NOAA's DOY Calendar. Or, on a system with Python installed, e.g., Linux or OS X, you can use the datetime module to obtain the day of the year corresponding to the current date as shown below:

$ python
Python 2.7.10 (default, Oct 23 2015, 19:19:21) 
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime
>>> exit()

Since today is October 7 of the year 2017, the day of the year is 280.

If you want to obtain the day of year (DOY) for another date, you can use, month, day).timetuple().tm_yday where year is the relevant year, month is the month, and day is the day of interest. E.g., March 1, 2017 is DOY 60:

>>> import datetime
>>>, 3, 1).timetuple().tm_yday

[/languages/python] permanent link

Sun, Oct 01, 2017 10:53 pm

Finding Gmail SMTP entries in Sendmail log files

I wanted to determine how many connections I was receiving per day from Gmail Simple Mail Transfer Protocol (SMTP) servers to my email server running Sendmail on a CentOS Linux system and the IP addresses of the Gmail servers that were sending email to users on my server. So I created a simple Python script to search for lines in the maillog file, /var/log/maillog for any lines containing "relay" and "" on the same line, since the Gmail servers are in Google's domain.

[ More Info ]

[/languages/python] permanent link

Tue, Sep 12, 2017 10:58 pm

Adding an item to a list in Python without duplicates

If you have a list in a Python script and wish to add new items to the list, you can use listname.append(newitem) where listname is the name of the list and newitem is the item you wish to add to the list. If you wish to add new items to the list only if they don't already exist in the list, you can avoid adding an existing entry to the list by checking if the entry is already in the list as shown below:

filenameList = []
if filename not in filenameList:

If you wish to know the length of a list, i.e., how many items are on the list, you can use print len(listname) where listname is the name of your list.

[/languages/python] permanent link

Fri, Sep 08, 2017 10:21 pm

Display a webpage from a Python script

If you just want to display text on a webpage from a Python script, a simple way to print to a web page is shown in the script below:


print "Content-type: text/plain\n"
print "Hello world"

If you want to display HTML code, you can use Python code like the following:


print "Content-type: text/html"
print """<!DOCTYPE HTML>
<html lang="en">
<h1>Hello World!</h1>

[ More Info ]

[/languages/python] permanent link

Sat, Aug 26, 2017 9:29 pm

Querying a MariaDB or MySQL database using Python

On a CentOS Linux system, I have a MariaDB (MariaDB is a fork of MySQL) database named "Books" that contains a table named "Amazon" where I've stored information on books that includes their Amazon Standard Identification Number (ASIN). I want to be able to run a Python script to which I provide a word as a command line argument and have the script find all books in that database table that have a title containing that word and to print the title, which is stored in a column named "Description" along with the ASIN, which is stored in a column named "ASIN". To do so, I use a script with the following code:


import mysql.connector, sys

def executeSQL(conn, queryTerm):
    cursor = conn.cursor()
    query = "select ASIN, Description from Amazon where Description like " + queryTerm
    result = cursor.fetchall()    
    for row in result:
       print row[0], row[1]

hostname = 'localhost'
username = 'myAcctName'
password = 'ThePassword'
database = 'Books'

except IndexError:
   print "Error - missing query word! Usage ./findBook queryWord"
   queryWord = "'%" + sys.argv[1] + "%'" 

connection = mysql.connector.connect( host=hostname, user=username, passwd=password, db=database )

[ More Info ]

[/languages/python] permanent link

Wed, Aug 23, 2017 11:36 pm

Determining a file's type from within a Python script

I needed a way to determine a file's type within a Python script when I can't rely on the file's extension to determine the file format. I'll be running the script on a MacBook Pro laptop running the OS X El Capitan operating system. OS X/macOS, like Linux, comes with the file command, so I could run that command at a shell prompt to have the utility check the magic number in the files I'm interested in, but I want to do some additional processing of the files within the Python script, so I want to perform the format check within Python. Python provides the subprocess module that provides the capability to "spawn new processes, connect to their input/output/error pipes, and obtain their return codes." So I can call the file utility from within Python using that module. To get the results from running a shell command, you use suprocess.Popen(). You can then set a variable to hold the results of .communicate() and print the contents of that variable as shown below. The script expects the name of the file to be checked to be provided as an argument on the command line.


import subprocess as sub, sys

except IndexError:
   print "Error - missing input file name! Usage ./ infile"
   fileName = sys.argv[1]

p = sub.Popen(['file',fileName],stdout=sub.PIPE,stderr=sub.PIPE)
output, errors = p.communicate()
print output

[ More Info ]

[/languages/python] permanent link

Mon, Aug 21, 2017 11:15 pm

Find all occurrences of a string in a file using Python

I need to download and view Excel workbooks, Portable Document Format (PDF), and sometimes other types of documents related to work requests from a website. When I view a webpage for a particular work request, some of the documents may have been posted weeks or months ago while some are more recent, but when I download them they all get the timestamp of the time I downloaded them, but I would like to change the timestamps to match the timestamps on the website. On the webpage for a particular work request, I see the following information for files:

Type Name Site Modified Modified By
xlsm icon CRQ000000884164_PDSO -None- 3/6/2017 4:53 PM Smith, Gary
pdf icon CRQ000000884164_PDSO -None- 3/6/2017 4:52 PM Smith, Gary
pdf icon CRQ000000884164_DCS -None- 6/12/2017 9:29 AM Doe, Mike
xlsm icon CRQ 884164_SDP -None- 6/12/2017 9:30 AM Doe, Mike

[ More Info ]

[/languages/python] permanent link

Fri, Aug 18, 2017 10:41 pm

Importing data from a text file into an SQLite database with Python

I need to track work requests that are associated with particular projects. The work requests are submitted through a website, but I want to be able to run queries on the data that aren't available to me from the website. So I decided to add a projects table to an existing SQLite database I have on my MacBook Pro laptop. I didn't want to manually enter a list of about 200 projects into the table, however, so I copied the projects list from the website into a text file. The webpage that lists all of the projects includes projects that are no longer valid and I want to include those in my database table, but note that they are longer valid. When I copy the data from the web page, I have lines like the following ones:

450 SN/GN Tech  Edit Delete
ACE     Edit Delete
ADO     Edit Undelete
AGO     Edit Delete
AGS     Edit Delete
AIM     Edit Delete
Artemis         Edit Undelete
ASF     Edit Delete
ATSC    Edit Undelete
AXAF    Edit Undelete
BATSE/Gro       Edit Undelete
CANDOS  Edit Undelete
CARA    Edit Delete
CD Manager      Edit Delete
CMF     Edit Undelete
CMOC    Edit Delete
CMS     Edit Undelete

The entries that are no longer valid have "Undelete" at the end of the line indicating the projects were deleted, but they can be undeleted.

I created a table in the database with the following columns all of which contain text data:

NameDescription AddedRemoved DeletedNotes

[ More Info ]

[/languages/python] permanent link

Once You Know, You Newegg AliExpress by

Shop Amazon Local - Subscribe to Deals in Your Neighborhood

Valid HTML 4.01 Transitional

Privacy Policy   Contact

Blosxom logo