Fussiness alone is not reason enough to try an elimination diet. Always talk with your doctor first. If your baby is reacting to something in your breast milk, like dairy, there are typically other symptoms like vomiting, diarrhea, rash, or weight loss. Your pediatrician can test for food allergies to confirm.