careful when casting numpy arrays

It's been a while since I posted here since I've been posting my questions (and eventual answers like this one) to stackoverflow.com.  In any case, I was running into a problem, that I thought I'd share with you all.

I have some code that was creating a numpy array from a list of lists. Depending on the user's interactions with our program, one of the columns in the list could contain strings, so I was casting the array to type np.object.  However, for very specific inputs, I was encountering a strange error:

ValueError: setting an array element with a sequence.

So I set out to take a closer look at what was happening and this is how I was able to reproduce the error:

>>>np.array(['asdf', 1.3456e-9]).astype('object')[1:].astype('float')
Traceback (most recent call last):
File "", line 1, in
ValueError: setting an array element with a sequence.

And more frighteningly:

>>> np.array(['asdf', 1.345678e-9]).astype('object')[1:].astype('float')
array([ 1.345678])

By this point, you probably see what's going on here.  Since I am _casting_ the array to type np.object, numpy is fitting all of the elements into equally sized blocks of memory that fit the largest string in the array.  The problem is that it doesn't seem to take the floating point values into account, so they are apparently cast to strings and tuncated when they are too large.  In the case above it appears that the underlying data representation would be numpy "|S8"since the float value coming back out is 8 characters wide.

In another example, we see that when the largest string is long enough, the whole float representation gets stored and properly casted.

>>>np.array(['asdfasdfasd', 1.34567e-9]).astype('object')[1:].astype('float')
array([ 1.34567800e-09])

The message here is never to cast a mixed type array to np.object. Rather, create it with that type to begin with so numpy knows how much memory to allocate for your floating point values.

>>> np.array(['asdf', 1.3456789123456e-9], dtype='object')[1:].astype('float')
array([  1.34567891e-09])

Matplotlib draggable legend

I thought I'd share my solution to the draggable legend problem since it took me forever to assimilate all the scattered knowledge on the mailing lists...

class DraggableLegend:
    def __init__(self, legend):
        self.legend = legend
        self.gotLegend = False
        legend.figure.canvas.mpl_connect('motion_notify_event', self.on_motion)
        legend.figure.canvas.mpl_connect('pick_event', self.on_pick)
        legend.figure.canvas.mpl_connect('button_release_event', self.on_release)
        legend.set_picker(self.my_legend_picker)
        
    def on_motion(self, evt):
        if self.gotLegend:
            dx = evt.x - self.mouse_x
            dy = evt.y - self.mouse_y
            loc_in_canvas = self.legend_x + dx, self.legend_y + dy
            loc_in_norm_axes = self.legend.parent.transAxes.inverted().transform_point(loc_in_canvas)
            self.legend._loc = tuple(loc_in_norm_axes)
            self.legend.figure.canvas.draw()
    
    def my_legend_picker(self, legend, evt): 
        return self.legend.legendPatch.contains(evt)   

    def on_pick(self, evt): 
        if evt.artist == self.legend:
            bbox = self.legend.get_window_extent()
            self.mouse_x = evt.mouseevent.x
            self.mouse_y = evt.mouseevent.y
            self.legend_x = bbox.xmin
            self.legend_y = bbox.ymin 
            self.gotLegend = 1
            
    def on_release(self, event):
        if self.gotLegend:
            self.gotLegend = False

#...and in your code...

def draw(self): 
        ax = self.figure.add_subplot(111)
        scatter = ax.scatter(np.random.randn(100), np.random.randn(100))
        legend = DraggableLegend(ax.legend())

update:
I emailed the Matplotlib-users group and John Hunter was kind enough to add my solution it to SVN HEAD.

On Thu, Jan 28, 2010 at 3:02 PM, Adam Fraser <adam.n.fraser@gmail.com> wrote:
> I thought I'd share a solution to the draggable legend problem since
> it took me forever to assimilate all the scattered knowledge on the
> mailing lists...

Cool -- nice example.  I added the code to legend.py.  Now you can do

leg = ax.legend()
leg.draggable()

to enable draggable mode.  You can repeatedly call this func to toggle
the draggable state.

Thanks,
JDH