Interdisciplinary criticism poses a problem: tribal loyalties may unite insiders, but outsiders may geninely lack the theoretical knowledge to evaluate the claims of insiders.
I propose a general experimental design to test the expert claims of an entire field: in this case, nutrition science.
The nutritionists' case is timely, as the field has attempted to enforce its monopoly on anything resembling nutrition advice by, for instance, trying to shut down bloggers who discuss nutrition but aren't "certified." A legitimate question for the field of nutrition studies is this: is the special knowlege obtained by formal education in this field effective in solving the most important nutrition problems?
The most salient, widespread, and harmful nutrition problem in industrialized Western countries is obesity. My experimental design will test whether nutrition science, as a field, offers effective solutions to obesity.
In my view, a solution is "effective" if (a) when used, it reliably produces weight loss in obese people; and (b) it is "doable" - people are capable of putting the solution into practice. So even though a lettuce-and-fish-tea diet might be shown to produce weight loss in those who stick to it, it's not effective if most people can't stick to it. The human mind itself, including its limited willpower, is relevant to the solution.
My experiment would test the claims of nutrition science as an entire field by measuring whether obese nutrition science students lose weight compared to matched controls enrolled in a program of comparable activity level.
An appropriate number of entering students in a nutrition science program (or more than one program) would be chosen as the experimental group. The control group would consist of a similar number of students enrolled in a similar education program in a subject unrelated to health or nutrition at the same institution(s), with similar demographics, overweight/obesity, and activity level.
Students in both groups would be weighed and have body fat percentages measured at the beginning of the experiment. Both groups would then have weight and body fat measured at the end of their programs, and perhaps five and ten years later.
The experiment is designed so that only the special knowledge varies between the experimental and the control group. (Suggestions welcome on how to better achieve this.)
If, at the end of five or ten years, the nutrition science students who were obese or overweight have lost weight (especially if they have become non-obese and non-overweight), and the healthy-weight students have not become obese, compared to the control group, that would be strong evidence that nutrition science provides effective solutions to obesity. Even if only 10% more obese nutrition science students than control students lost weight, that would be evidence that nutrition science has a true understanding of obesity that can be translated into real-life solutions.
However, if at the end of five or ten years, the nutrition science students have not lost more weight or remained healthier than control students, that would be some evidence that nutrition science, as a field, does not truly understand obesity in a way that can be translated into effective solutions, as define above.
This experimental design is not limited to nutrition science, of course. It could equally well apply to, for instance, therapeutic psychology; if psychology students become less depressed than control students, that's evidence of true understanding and effective methods; if psych students are not less depressed than control students at five or ten years, that would be some evidence that the special knowledge of psychology does not constitute a deep understanding of the problems it purports to address.
Thanks to @wonkinakilt on Twitter for suggestions!