Avoid double panics when using `TempDir` in tests `TempDir` could panic on drop if `remove_dir_all` returns an error. If this happens while already panicking, the test process would abort and therefore not show the test results. This PR tries to avoid such double panics.