This paper is devoted to the solving an eigenvalue problem for opened layered cylindrical waveguide structure with arbitrary finite number coaxial magnetodielectric layers. Classical method of separation of variables for analitycal solution to a boundary value problem for second order non-self-adjoint differential operator is applied. A general solution for electromagnetic field components, its boundary and infinity conditions are applied to obtain the complex transcendental dispersion equation. The dispersion equation has form a condition of nontrivial solving for system of homogeneous linear equation, that is a condition of singularity for a matrix D of the system linear equation, for example det(D)=0. For l-layered waveguide structure this is a square 4(l+1)-matrix, which rank not exceed 4l+3. If a rank of matrix D is equal 4l+3, then simple roots of dispersion equation exist, else multiplicity of roots to appear. The matrix D can be represented in block form respectively to vectors of unknown complex amplitude coefficients. Blocks of submatrices are bidiagonal.
The dispersion equation solutions for the particular waveguide structure represents the longitudinal wavenumber values of wave modes and a variety of wavenumber values represent discrete mode spectra: surface modes and leaky modes. In general this is hybrid modes excepting a case of axial symmetrical field distribution. For this case the dispersion equation is decomposed to two equation – for TE-modes and for TM-modes and the matrix D obtain a block diagonal form.