Okay I did some testing again and you are right it seemed to work without the SkeletonGraphicCustomMaterials component. Maybe something else was going on that was making it not work, as I was on a different git branch at the time so might have messed something up. Thanks for the confirmation though!
By the way, is it recommended to interact with SkeletonGraphic after Start() or doesn't matter? On UI that is initially inactive, the SkeletonGraphic will obviously not have Awaked/Started, until the UI that it is on is enabled.