Here I've added the second spacer.  I made it as large as possible to get the best stability when the skeg is sliding in the trunk.  The flat edge on the bottom of the spacer forms a line that runs between the bungee notch and the tangent to the axle.  I experimented with a full disk (looking like a rudder pulley) with the bungees riding along the lower edge, but it changed the direction of force of the bungee and the skeg would slip off the axle.   The current configuration allows the bungee to move freely across the surface of the skeg without the chance of binding between the spacer and the trunk..

The bungee is slightly tensioned  with the skeg is it fully extended position.  With the pivot point where its at, the bungee will be gradually stretched as the skeg is moved into its retracted position by the up-haul cable.

Since the bungee anchor point is easily accessible, it will be easy to adjust the tension to get it just right.  The bungee pulls the skeg against the axle and stop-block, but the skeg can be easily removed by stretching the bungee and sliding the skeg off the axle.