Hamiltonian Monte Carlo (HMC) algorithms are widely used to generate samples from a given probability distribution. They are based on numerically integrating a Hamiltonian differential system, with the leapfrog/Verlet integrator being the integrator of choice. This integrator is based on splitting the Hamiltonian into its potential and kinetic parts. Often, probability distributions may be seen as a perturbation of a Gaussian. When using HMC algorithms to sample from those targets, it is tempting to alternatively split the Hamiltonian $H$ as $H_0(\theta,p)+U_1(\theta)$, where $H_0$ is quadratic and $U_1$ small and perform the required numerical integrations of the Hamiltonian dynamics by combining integrations for $H_0$ and integrations for $U_1$. This idea is appealing because, if $U_1$ were to vanish, the integration would be exact so that it may be hoped that for small $U_1$ the integration would be easy to perform. We will show that, unfortunately, samplers based on the $H_0+U_1$ splitting suffer from stepsize stability restrictions similar to those of algorithms based on the standard leapfrog integrator. The good news is that those restrictions may be circumvented by preconditioning the dynamics. Numerical experiments show that, when the $H_0(\theta,p)+U_1(\theta)$ splitting is combined with preconditioning, it is possible to construct samplers far more efficient than standard leapfrog HMC.