Ants are attracted to the sticky honeydew secreted by aphids, leafhoppers, mealybugs, whiteflies, and scale insects. While not problematic in and of itself, the presence of ants in your garden may indicate and/or exacerbate aphid problems. See Aphids, Hoppers, Mealybugs, Scales and Whiteflies.
Specific recommendations for fruit trees
To prevent ants from entering the foliage crowns of fruit trees, where they may aggravate aphid problems, use a 4-inch wide strip of polyester fiber matting to snugly surround the trunk, covered by a 4-inch wide strip of household plastic wrap. Smear this with a sticky substance such as Ant-Bar. The barrier will stretch as the tree grows, but once it splits, replace it.
Ants & Peonies
Many gardeners notice ants on peony buds and assume that ants are needed for the delicate blossoms to open, however, that is not the case. Ants are almost always after sweet treats, and in this case, they are after the extrafloral nectaries that are located outside of the flower buds.