The term “Worthies” was often used in Victorian times in relation to notable, admirable, or historically significant individuals. However, with the passage of time, it is not clear if the term was also used less positively, in this case marking out individuals who were notably different or considered a bit “odd”.
I would not feel comfortable using this term today.
The following images are from the Tain & District Museum:
North Star and Farmers’ Chronicle, 1 January 1949: