This paper presents a general technique for the joint Direction-of-Arrival (DoA) and Time-of-Arrival (ToA) estimation in multipath environments. The proposed ultra-wideband technique is based on phase-mode expansions and the use of nearly frequency-invariant elliptical arrays. New possibilities open with the present approach, as not only elliptical, but also circular and linear arrays can be considered with the same implementation. Systematic selection/rejection of signals-of-interest/signals-not-of-interest in smart wireless environments is possible, unlike with previous approaches based on circular arrays. Concentric elliptical arrays of many sizes and eccentricities can be jointly considered, with the subsequent improvement that entails in DoA and ToA detection. This leads to the realization of pseudo-random array patterns; namely, quasi-arbitrary geometries created from the superposition of multiple elliptical arrays. Some simulation and experimental tests (measurements in an anechoic chamber) are carried out for several frequency bands to check the correct performance of the method. The method is proven to give accurate estimations in all tested scenarios, and to be robust against noise, channel nonidealities, position uncertainty in sensor placement and interferences.