Stephen Foster wrote the state song, My Old Kentucky Home. Federal Hill, with an inscription that reads, My Old Kentucky Home. A thoroughbred racehorse is positioned behind a fence in the foreground of the quarter. Kentucky was the first state on the western frontier to join the Union and is one of four states to call itself a commonwealth. Kentucky is home of the longest running annual horse race in the country, the Kentucky Derby.