We have been to Wombeyan caves also, and had a great time with our kids. You can take a (fairly long) walk along a marked trail and end up at the bottom of a gorge with beautiful water to splash and swim in. We were also able to have a real campfire, with safe concreted areas to have the fire in. Our kids were about 5 and 7 at the time and had no trouble in the caves we went to.
We will definitely go back there. A word of warning, the road to get there from Mittagong is about 40 km of winding dirt track on edge of mountains. Give the kids travel sick medicine if needed!!