Samtskhe-Javakheti is a region of Georgia, which borders Northern Armenia to the south.
Samtskhe-Javakheti used to be known as Meskheti, for its Meskhi Georgian tribes. The region is especially dry and mountainous south of Borjomi (which is still as green and wet as anywhere in Imereti). Javakheti, the southeastern portion of the region around Akhalkalaki, sees a bit of political ferment, since the vast majority of its residents are ethnically Armenian and demand greater autonomy and language rights. Unlike in Abkhazia and South Ossetia, however, the political situation here does not translate into any danger for a visitor; it is rather just an extra reason why Javakheti is an interesting destination.
Visitors should make Vardzia their number one travel priority (and stop along the way at Khertvisi Castle). but those with some extra time to explore would be foolish not to haul themselves up the bumpy mountain road to see the treasure that is Sapara Monastery. Borjomi is another great destination, but is far easier to visit than the rest of the regional attractions, as it is located very close to the main highway going between Tbilisi and Kutaisi.
In the southeast of the region around Akhalkalaki, Armenian is widely spoken, even more so than Georgian. But in the rest of the region, Georgian language speakers predominate. Russian is also widely spoken, especially by older generations.
Marshrutkas are generally the best when travelling south of Akhaltsikhe, as they seem to handle the bumpy dirt roads better than the Soviet built taxis.
For connections in the southern part of the region, Akhaltsikhe, Akhalkalaki, Ninotsminda and even Gyumri (Armenia), checkout Akhaltsikhe#By marshrutka. Also, there are three marshrutkas from Didube Station in Tbilisi to Akhalkalaki: at 10:00, 12:00, and 18:00, but it's not clear which route they take (via Borjomi or direct).
Besides the famous Vardzia (a 12th-century cave monastery and city carved out of a cliff overlooking a river gorge), there are Upper Vardzia, Khertvisi Castle and Vanis Kvabebi (another cave monastery) closeby. See all the information at the Vardzia article.
Furthermore, near Akhaltsikhe do not miss the interesting Sapara Monastery.
Going deeper into the Vardzia valley, there seem to be some nice hiking opportunities going up from the valley. Also, the closeby monasteries of Akhaltsikhe invite for hiking and camping.
Check out the thermal bath near Aspindza:
... and on the way to Vardzia:
If you are looking to sample some Armenian cuisine, Akhalkalaki is home to some authentic-as-it-gets cheap Armenian cafes.
Samtskhe-Javakheti is a quite safe region of Georgia. Probably the biggest danger to a visitor is the risk of falling off various cultural monuments—there are no guard rails to speak of. But common sense and a natural fear of heights should keep you safe.