Fad diets work because they make it hard to find food
Every few years, a new diet becomes popular: Atkins, then Paleo, then Keto, then Gluten-free, each one developing a following of people who swear the diet helped them lose weight. My unsubstantiated theory: while each diet makes grand claims about its underlying science, I suspect these diets work, at least at first, for a much simpler reason: they make it hard to find food.
I am a reductionist when it comes to weight loss: if your calorie intake is less than what you consume, you’ll lose weight; otherwise, you’ll gain weight. While different foods or routines might make it easier to maintain a calorie deficit, the basic principle is the same: you have to eat less than you use.
When a new diet comes with strict requirements - for instance prohibiting gluten - it makes it harder to find food that you can eat. Followers of the diet may go to a restaurant and find that it serves very few gluten-free entrees, or that when they reach for an afternoon snack nothing they want fits the criteria. So, followers of the diet end up eating less, thereby losing weight.
However - as the diet becomes more popular, more restaurants and manufacturers start catering to the diet’s restrictions. For example, in the SF Bay Area it’s now fairly common to see gluten-free pasta, pizza, crackers, and cookies. As it becomes easier and easier to find food that meets the diet’s criteria, people following the diet end up consuming nearly the same number of calories as they would not on the diet, and the diet loses its effectiveness. Another diet comes along claiming some different restriction, which again works for a while by making it hard to find compatible foods, and the cycle repeats.
This is of course pure speculation by an untrained amateur, but it seems to explain why we see the continued cycle of people claiming effectiveness of scientifically-questionable diets.