Sure, Why not? The main problem is that in most places in the state climbing is prohibited, either because the locations are private property or because they are in State Parks or just because. There are, however, a few places in the state where a rock climbing fan in need of a quick fix can get their rocks off.
Location #1 is called Unlikely Wall. It's a road cut at the Harrodsburg-Lake Monroe exit off of Indiana State Road 37 about 15 miles south of Bloomington. It has a number of routes and is rated in difficulty from 5.4 to 5.12 on the Yosemite Decimal System. (Forgive any gross stupidities here, I know nothing about rock climbing). The cliff is located north of the exit and west of the highway. You are supposed to park underneath the overpass.
Locations #2-5 are all in the area of Shoals, Indiana. To reach Shoals, take Highways 50 and 150 east from Highway 231 in Loogootee. On the way in you will pass the interesting formation Jug Rock (not to be climbed). Formations that may be climbed in this area are the House Rock, McBride's Bluff and Riverside Bluffs, all on or near White River Road which goes North from Shoals and follows the White River (!). South and west of Shoals on Spout Spring Road leads to Road 550 and Hindoostan Bluffs just past The State Fishing Area. All areas mentioned have at least a couple of routes and McBride's Bluff has a bunch.
I recommend checking out a book, if still (self-)published, called Rock'n'Indiana, by Kevin Foy. Kevin also does instruction and lead climbing; the book lists his number as (317)846-6457. I got my copy at JL Waters and Company of Bloomington, Indiana. This book contains a whole lot of rock-climbing gobbledegook that may possibly be comprehensible to other rock-climbers, and suggestions about routes. Have fun, and try not to die.