I'd suggest turning a boss on the end of a faced short steel bar in the lathe, and bore the ID of the boss. The bar needs to have a diameter a little more than twice the distance from centre to the end of the arm of course. 1.125" dia stock or larger would work for the arm in the drawing. Then on the mill, with the rod set vertically, cut most of the disc away leaving the boss and the arm shape, and drill the end hole for the pin. After that you could slice the arm off the end of the rod with a slitting saw. The pin could then be made in the lathe and silver soldered in to the arm, and the setscrew hole drilled and tapped.
For this type of soldered-in pin, I usually make them with a small thin head, to keep the pin in place easier during soldering. After the soldering, the head can be left on if there's room, or filed away if there isn't room.
Hope these ideas help.