lundi 13 juin 2016

Python List of Lists vs numpy

So, I have a snippet of a script:

lol = []
latv1 = 0
latv2 = 0
latv3 = 0

#Loop a
for a in range(100):

    #Refresh latv2 after each iteration of loop a
    latv2 = 0

    #Loop b
    for b in range(100):

        #Refresh latv3 after each iteration of loop b
        latv3 = 0

        #Loop c        
        for c in range(100):

            #Make 4 value list according to iteration and append to lol
            midl2 = [latv1,latv2,latv3,0]
            lol.append(midl2)

            #Iterate after loop
            latv3 = latv3 + 1
        latv2 = latv2 + 1
    latv1 = latv1 + 1

Which will do what I want it to do.... but very slowly. It gives:

[[0,0,0,0]
 [0,0,1,0]
 ...
 [0,1,0,0]
 [0,1,1,0]
 ...
 [9,9,8,0]
 [9,9,9,0]]

I've read about numpy and its speed and optimization. I cannot figure out how to implement with numpy what I have above. I've learned how to make an array of zeroes with numpy via the manuals:

numpy_array = np.zeroes((100,4))

To give:

[[ 0.  0.  0.  0.]
 [ 0.  0.  0.  0.]
 [ 0.  0.  0.  0.]
 ..., 
 [ 0.  0.  0.  0.]
 [ 0.  0.  0.  0.]
 [ 0.  0.  0.  0.]]

and can change the values of each column with:

numpA  = np.arange(0,100,1)
numpB  = np.arange(0,100,1
numpC  = np.arange(0,100,1)
numArr[:,0] = numpA
numArr[:,1] = numpB
numArr[:,2] = numpC

giving:

[[   0.    0.    0.    0.]
 [   1.    1.    1.    0.]
 [   2.    2.    2.    0.]
 ..., 
 [ 997.  997.  997.    0.]
 [ 998.  998.  998.    0.]
 [ 999.  999.  999.    0.]]

but I cannot create a numpy array 1000000 lines long and have the columns increment like the original example did. If I call the zero array creation with 1000000 instead of 100 the column substitution does not work, which makes sense as the length of the array and the substitution are unequal - but I am not sure how to correctly iterate the substitution arrays to work.

How can I replicate the original scripts output via numpy arrays?

Note: This is a python 2.7 machine, but it's 64 bit at least. I know RAM use is an issue, but I should be able to change the dtype of the array to fit my needs.

Aucun commentaire:

Enregistrer un commentaire