Vectorizing images in Python for Machine Learning image classification

By


I'm doing the Astrozoo competition on Kaggle right now. The goal is to create an algorithm to accurately classify galaxies into different types. The training set is a zip file consisting of over 50,000 images. Before you do anything, the first step would be to convert the images into a usable format for analysis, such as resizing each image to 100x100 and converting it into a vector of data points. Here is the code I've pieced together to do that.

Running this code will convert all jpeg files in a specified folder into vectors and dump it into a .txt file.







#take all files in one directory and move it into another directory

import os
import glob #allows you to look at file names
import PIL #allows you to manipulate images (for resize)

#main script variables

orig_dir = '/home/luwei/Desktop/Dropbox/Kaggle/astrozoo/testing' #where the images are located
crop_dimensions = (140, 140, 284, 284) #dimensions to crop each image by
pixelsize = 5 #set size of pixels for the pixelate operation
vectorfile = '/home/luwei/Desktop/Dropbox/Kaggle/astrozoo/imagevectors.txt' #directory of vector output file



#change the directory to where the images are 
os.chdir(orig_dir)


filenames = glob.glob("*.jpg") #store all the filenames into an array


#resize every image in the folder to 
with open(vectorfile, 'wb') as outfile:
    for fileid in filenames:
        img = Image.open(fileid) #open the image
        img = img.crop(crop_dimensions) #crop each image
        img = img.resize((100,100)) #resize the image 
        img = img.convert('L') #convert each image to grayscale 
        ###pixelate the image###
        img = img.resize((img.size[0]/pixelsize,img.size[1]/pixelsize), Image.NEAREST) #reduce the size of the image by factor of 5
        img = img.resize((img.size[0]*pixelsize, img.size[1]*pixelsize), Image.NEAREST) #increase the size of the image by a factor of 5
        #########################
        pixel = img.load() #store pixel data in the pixel variable 
      
        imagevector = [] #create an empty imagevector for each image
        u = 0 #marker for the pixel 
        
        for x in range(0, img.size[0], pixelsize): #for every x coordinate in the pixel data of the image
            for y in range(0, img.size[1], pixelsize): #for every y coordinate in the pixel data of the image
                pixelvalue = pixel[x,y] #value of that particular pixel coordinate
                imagevector.append((u, (x, y), round(pixelvalue/float(255),3) ) ) #store pixel number, coordinate, and grayscale value (converted from 0 to 255 to a number between 0 and 1)into an array
                u = u + 1 #add one to the pixel marker
        
        vectorfile = open('imagevectors.txt', 'r+')
        outfile.write(str(fileid) + " " + str(imagevector) + "\n")
        print fileid
    
        #img.save('resized' + file + '.jpg')








































Comments











Post a Comment






























Popular posts from this blog









grandmaster level chess AI using python - Part 2 (the code) 







By





















Image







In continuation of part 1, I will now building my chess AI in python.  As mentioned before, my goal is to create a chess AI at the Grandmaster level (elo > 2500)   Here is the code for the chess AI.  https://github.com/luweizhang/chess_ai   Key components of the chess game / AI:   Python Classes:   ChessGame   instantiates the chessboard and pieces  keeps track of whose turn it is   RulesEnforcer   Logic for how all the pieces move, castling   Makes sure you cant make illegal moves  Used in the chess AI to generation possible positions    ChessAi   Tree generator   generates the tree of position moves   Position Evaluator   evaluates the quality of chess positions   Minimax   Determines the optimum move from the chess tree    ChessDB   Opening database         Explanation of the tree generator:   The tree generator component of the chess AI generates the possible future board positions based on the possible moves that the pieces can make.  At any given position, there are o...








Read more










building a chess ai - part 4: learning an evaluation function using deep learning (keras)







By





















Image







Note: this post is still in progress.   Below I will discuss approaches for training a deep learning chess game and the results of my implementation.   Approach 1: Train against the outcome of the game  *y*  is the outcome of the game (1 is win for white, 0 is loss for white, .5 is draw)  We want to learn a function * f(p)*  that can approximate this.   *p*  is the chess position (8x8 chess board), actually is an 8x8x12 = 768 dimensional vector (since there are 12 pieces)      8x8x6  = 384 dimensional vector with positive values for squares with a white piece and negative values for squares with a black piece.  (Using 64 x 12 introduces too many degrees of freedom)   The goal is to try to learn a function * f(p)*  that can predict the winner of the game * y*  given a chess position * p* .   The model:   Dataset:   http://www.ficsgames.org/download.html  millions of high quality games played by grand masters or international masters   Objective/Loss Function...








Read more










Brief intro to recurrent neural networks







By





















Image







Note: This post is meant to be a brief and intuitive summary of recurrent neural networks.   I have adapted this material from the Coursera deep learning course.  The value I hope to add here is that I have attempted to summarize the information in a way that is easy (hopefully) to understand, and can be used as reference or refresher material for the future.    Part 1: Recurrent Neural Networks:   Recurrent neural networks are a class of neural networks where the nodes/neurons form a directed graph along a sequence.  They are very effective at tasks such as Natural Language Processing because they have a "memory," in other words they can receive context from previous inputs.  They can take in input one at a time, and can pass information from one node to the next via hidden activation layers.  This information serves as the "memory" or "context" layer of the network, which can be used in conjunction as more new input is being processed.    RNN Cell:      ...








Read more