How to download images from s3 as numpy arrays?

I am trying to train a neural network where I pass in a series of images. I want to create a generator which passes each image in as a numpy array

from skimage import io
image_array = io.imread(url)

I want something like this:

s3 = boto3.resource('s3')
my_bucket = s3.Bucket('some-bucket')
def my_generator():
    for object in my_bucket.objects.all():
        image_array = io.imread(object)    # this will not work. object is of type s3.ObjectSummary(bucket_name='manga-learn-data', key=u'one-piece-colored-5340113_06_05.png') whereas io.imread is expecting a url or uri. 
        yield image_array

But that image_array variable will never work. Everything I can find about downloading images from amazon s3 indicates that you download the file to a file. I want to download it to an image object that I can open as an array.