That sounds right. Some plate screws have a major diameter and a small section of minor diameter just under the screw head. The plate should be drilled to fit the larger diameter below the shoulder with a split washer or wavy washer taking up the space of the smaller diameter just under the screw head. You should try putting a small, light duty split lock washer or a light wavy washer in between the screw head and the plate to keep the plate pivoting on the larger diameter, below the shoulder. Of course if that is the situation?