The Central South Eastern Football Association was formed in 1924 with the foundation clubs being Frances, Lucindale, Narracoorte and Penola. The member clubs competed for the Caldwell Challenge Cup.
Penola and Narracoorte decided to leave after the 1925 season as they went in search of stronger competition. The competition lapsed in 1926 and then reformed in 1927 with four different clubs, Lucindale, Kybybolite and two South East association clubs, Penola and Narracoorte.
Penola won the 1924 flag using its right to challenge as the beaten minor premier. The Caldwell trophy was presented to the captain Len Redman and then soon after the trophy mysteriously disappeared. More than eighty years later the trophy was found by chance in Gregory, a Western Australian coastal town and returned to the club.