Jquery UI Sortable Grid Pinned Items

I've been spending all day trying to figure out how to make a jquery ui sortable grid, with pinned / sticky / locked items.

Searching around on stack and google has gotten me this far:

 $(function() {
    $("#sortable").sortable({
        start: function () {
            $(this).find(".pinned").each(function () {
                $(this).data("fixedIndex", $(this).index());
            });
        },        
        change: function () {
            $(this).find(".pinned").each(function () {
                $(this).detach().insertAfter($("#sortable li:eq(" + ($(this).data("fixedIndex")-1) + ")"));
            });
        }
    });
});

https://jsfiddle.net/mvirik/9ear1n3o/17/

This seems to work fine, but if I add "pinned" class to first or last item, it all breaks down.

Any help would be really great. Thank you!