Helicopter design improvements typically include higher speed as one of the goals, along with thrust capability for operations in challenging conditions like low air density. Apart from performance, aeromechanical stability of the rotor is also an important consideration under these flight conditions. Design process for such helicopters will require an assessment of performance and stability with inclusion of i) aerodynamic model suitable for high advance ratio ii) efficient computation of damping in forward flight conditions iii) experimental investigations of the rotor (at least in hover) to verify the correctness of the aerodynamic coefficients of the airfoil. A coaxial rotor is a good candidate for high speed operations as well as for operation in low density atmosphere as found in Mars. The thesis demonstrates the use of some of the improvements described below in modeling, trim and stability analysis towards a coaxial rotor design for low density operation. The stability of a helicopter rotor in forward flight depends on several factors such as aerodynamics, blade structural parameters, and control inputs. In high speed forward flight, most of the blade sections encounter three-dimensional effects due to the radial component of relative air velocity. These three-dimensional effects can have a significant influence on the airfoil aerodynamic characteristics depending on the angle of attack of the blade section and hence can affect the stability of rotorcraft. Specifically, the effect of yawed flow on lead-lag mode damping of an isolated rotor in forward flight is investigated in this work. The damping results are also correlated with existing experimental data. Correlation results improve at high advance ratio when yawed flow effects are included in the aerodynamic model. When linearized, the forward flight system is periodic corresponding to rotation frequency of the blades, and stability computations hence rely on Floquet theory. Floquet transition matrix (FTM) is usually estimated by integrating the linearized system of equations using n (number of states) independent initial conditions. The stability of the rotor is then evaluated from the eigenvalues of FTM. The limitation of this classical approach are 1) Integration with an optimization problem, 2) Using time-varying control inputs (when used with periodic shooting), 3) Computational cost to estimate the Floquet eigenvectors for mode identification as well as periodic control application. Stability and trim analysis based on a pseudospectral method based framework are demonstrated for rotorcraft analysis to address the above limitations. Further, the framework is adapted to include the concept of fast-Floquet which relies on blade-to-blade symmetry to further improve the computational efficiency. In addition, the pseudospectral framework is well suited to include a dynamic inflow model for coaxial rotors. The framework described above is utilized for solving an optimization problem. For rotors operating in low air density (like that of Mars), the aerodynamic damping experienced by blades is significantly reduced. Hence, it is beneficial to estimate the operating forward speed of rotorcraft with stability as a constraint during design stage. Experimental work is also carried out for measuring thrust on a rotor fixed to a test stand inside a low-density chamber. The purpose of the experimental work is to validate the basic blade aerodynamic model in hover. A preliminary aerodynamic design is presented for a coaxial rotorcraft based on blade element momentum theory. Forward flight trim analysis for the designed coaxial rotor is performed using pseudospectral framework with inclusion of dynamic inflow models