PandasIndex init with object indices slow without cftime #5929

What happened:
While profiling some basic code working with datasets, I came across a repeated import that was slowing things down considerably. During various dataset operations, PandasIndex instances are initialized. When an index is object-valued, PandasIndex.__init__ calls safe_cast_to_index -> _maybe_cast_to_cftimeindex -> CFTimeIndex.__new__ -> assert_all_valid_date_type -> import cftime. If cftime is not installed, this generates a ModuleNotFoundError, which is caught in _maybe_cast_to_cftimeindex. However, searching and failing to find the cftime import takes much longer than correctly finding it. As a result, this code is much slower if cftime is not installed.

What you expected to happen:
PandasIndex.__init__, as a relatively commonly called portion of code, should be reasonably fast. Its execution time should not strongly depend on an optional dependency like cftime.

Minimal Complete Verifiable Example:

import xarray as xr
from xarray.core.indexes import PandasIndex
import pandas as pd
from timeit import timeit

print(timeit(lambda: PandasIndex(pd.Index(data=['abc', 'def'])), number=1000))
# with cftime, this takes about 60 ms
# without cftime, this takes over 4000 ms

Anything else we need to know?:
Relevant code is here: