After you confirm the warning message, File|Delete closes the active file and also deletes it from disk.

If Windows keeps a Recycle Bin for the drive the active file is on, it will be moved into the Recycle Bin awaiting its final fate.

See also

File Menu
Project|Delete Project