Unlike neighboring countries, Germany is very decentralized. While, when in France, one would say you've got to see Paris, and in England, you'd say the same about London, and in Spain it would be Barcelona, Madrid, there is really no place one has got to go in Germany, and get it all in one. That certainly goes for the large cities, but similarly for the smaller towns or regions. Best you try and see it all, but don't expect to do that in just a few weeks. Germany is not a really large country but it is so full of things worthwhile seeing, that you will at best get to see only a small portion even in a few months.
There are so many nice places in Germany that are absolutely underrated and unknown. Among the larger and better known towns, I could recommend places like going from north to south Lubeck, Schwerin, Rostock, Rügen, Münster, Wernigerode, Quedlinburg, Erfurt, Potsdam, Eisenach, Göttingen, Trier, Idar Oberstein, Speyer, Schwetzingen, Heidelberg, Ladenburg, Baden Baden, Strassburg (in German speaking France :-)), Bamberg, Rothenburg ob der Tauber.
Among the Regions that are great are: Wattenmeer (North Sea Coast), Mecklenburger Seenplatte (East Germany), Harz Mountain Range, Weserland (river), Münster, Rhine Valley from Bonn to Mainz, Black Forest, Allgäu incl. Neuschwanstein, and Bodensee.
In between all of this, there are so many thousands of great places and things to see. I don't believe that you can go for more than 10 miles without seeing anything. Two ways to get around Germany, if you REALLY want to see the underrated places: Bicycle and Train, or a combination thereof. Get a Eurrail pass, and you can essentially go everywhere for free. Stay over night at pensions or garni hotels, and you will save a lot of money.