It simply means working out the amount of calories you can consume per day (protein, fat and carbs), and not going over that number - regardless of what you eat. If your limit was 3000kcal per day, as long as you don't go over that number, then it's fine. This means you could eat healthy foods or junk, but 3000 is 3000. Some people do use the phrase with the only intent of eating healthy foods, whereas I have seen posts on social media showing a McDonald's meal with a comment of "IIFYM, this does, all good!". The number of calories in a fast food meal and a healthy meal could be the same, but the quality of those calories are very different.
It is good to work out your calorie expenditure and the amount of calories you need to consume for your goals, but don't think 'a calorie is a calorie' and as long as you don't go over your limit, you're fine, because you're not. You need to keep the calories as healthy as possible to see the results that you want.
The main thing you need to focus on is eating healthy and keeping things consistent. Yes, you want to stick to a calorie limit, but that doesn't give you freedom to eat what you want as long as you stay on that number - quality first!
Lee Gregory Fitness
(picture is not mine)