For anyone else looking for this:
In the first case the query was actually selecting the right posts – but the KML maps in each post were small and widely separated so it was difficult to see them the mashup map.
In the second, with two categories using “category_name=a,b” there is a bug in MapPress that is causing them to not be selected properly.
The bug will be fixed in 2.38.8. If you have the same issue, as a workaround you can use “category__in=1,2” instead. Note that this requires category ids instead of category names, and does not include the children of the category.