buildman: Fix the logic for the bloat command

This check should now be done whatever mode buildman is running in, since
we may be displaying information while building.

Signed-off-by: Simon Glass <sjg@chromium.org>