That is a BIOS setting, you have it set to use the HDD as first boot device. You can either go into your BIOS setting to adjust that and make USB the first boot device, HDD second (usually hold the Esc or Delete key during boot up to get to the BIOS). Or use the assigned F key to select what device to boot from, this is usually F8 or F11, some systems F12.