We consider propagation of surface TM waves in a circular dielectric waveguide filled with nonlinear (Kerr nonlinearity) multilayered inhomogeneous medium. Each layer is characterized by a specific value of the nonlinearity coefficient α. Analysis is reduced to solving a nonlinear transmission eigenvalue problem for an ordinary differential equation; eigenvalues of the problem correspond to propagation constants of the waveguide. For the numerical solution, a method is proposed based on solving an auxiliary Cauchy problem (a version of the shooting method). As a result of comprehensive numerical modeling, new propagation regimes are discovered.