lundi 27 juin 2016

Permute rows in "slices" of 3d array to match each other

I have a series of 2d arrays where the rows are points in some space. Many similar points occur across all arrays but in different row order. I want to sort the rows so they have the most similar order. Also the points are too different for clustering with K-means or DBSCAN. The problem can also be cast like this. If I stack the arrays into a 3d array, how do I permute the rows to minimize the average standard deviation (SD) along the 2nd axis? What's a good sorting algorithm for this problem?

I've tried the following approaches.

  1. Create a set of reference 2d array and sort rows in each array to minimize mean euclidean distances to the reference 2d array. This I am afraid gives biased results.

  2. Sort rows in arrays pairwise, then pairs of pair-medians, then pairs of that, etc... This doesn't really work and I'm not sure why.

A third approach could be just brute force optimization but I try to avoid that since I have multiple sets of arrays to perform the procedure on.

This is my code for the 2nd approach (Python):

def reorder_to(A, B):
    """Reorder rows in A to best match rows in B.

    Input
    -----
    A : N x M numpy.array
    B : N x M numpy.array

    Output
    ------
    perm_order : permutation order
    """

    if A.shape != B.shape:
        print "A and B must have the same shape"
        return None

    N = A.shape[0]

    # Create a distance matrix of distance between rows in A and B
    distance_matrix = np.ones((N, N))*np.inf
    for i, a in enumerate(A):
        for ii, b in enumerate(B):
            ba = (b-a)
            distance_matrix[i, ii] = np.sqrt(np.dot(ba, ba))

    # Choose permutation order by smallest distances first
    perm_order = [[] for _ in range(N)]
    for _ in range(N):
        ind = np.argmin(distance_matrix)
        i, ii = ind/N, ind%N
        perm_order[ii] = i
        distance_matrix[i, :] = np.inf
        distance_matrix[:, ii] = np.inf

    return perm_order


def permute_tensor_rows(A):
    """Permute 1d rows in 3d array along the 0th axis to minimize average SD along 2nd axis.

    Input
    -----
    A : numpy.3darray
        Each "slice" in the 2nd direction is an independent array whose rows can be permuted
        to decrease the average SD in the 2nd direction.

    Output
    ------
    A : numpy.3darray
        A with sorted rows in each "slice".
    """
    step = 2
    while step <= A.shape[2]:
        for k in range(0, A.shape[2], step):

            # If last, reorder to previous
            if k + step > A.shape[2]:
                A_kk = A[:, :, k:(k+step)]
                kk_order = reorder_to(np.median(A_kk, axis=2), np.median(A_k, axis=2))
                A[:, :, k:(k+step)] = A[kk_order, :, k:(k+step)]
                continue

            k_0, k_1 = k, k+step/2
            kk_0, kk_1 = k+step/2, k+step

            A_k = A[:, :, k_0:k_1]
            A_kk = A[:, :, kk_0:kk_1]

            order = reorder_to(np.median(A_k, axis=2), np.median(A_kk, axis=2))
            A[:, :, k_0:k_1] = A[order, :, k_0:k_1]

        print "Step:", step, "t ... Average SD:", np.mean(np.std(A, axis=2))
        step *= 2

    return A

Aucun commentaire:

Enregistrer un commentaire