This term is stipulated in a specification of the California Department of Transportation, who first started using concrete median barriers in the mid-1940s. The reason for the letter K is uncertain.
K-rail (plural K-rails)